-- Logs begin at Wed 2022-05-25 15:26:04 UTC, end at Wed 2022-05-25 16:39:52 UTC. -- May 25 16:05:07.497601 dv-cinder-843278-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 25 16:05:07.507282 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 25 16:05:07.508693 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed May 25 16:05:07 2022] *** May 25 16:05:07.508693 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 25 16:05:07.508693 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 25 16:05:07.508693 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: nodename: dv-cinder-843278-2-iscsi May 25 16:05:07.508693 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: machine: x86_64 May 25 16:05:07.508693 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: clock source: unix May 25 16:05:07.508693 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: pcre jit disabled May 25 16:05:07.508693 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: detected number of CPU cores: 4 May 25 16:05:07.508693 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: current working directory: / May 25 16:05:07.508693 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: detected binary path: /usr/bin/uwsgi-core May 25 16:05:07.509741 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: your processes number limit is 47680 May 25 16:05:07.509741 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: your memory page size is 4096 bytes May 25 16:05:07.509741 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: detected max file descriptor number: 2048 May 25 16:05:07.509741 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: lock engine: pthread robust mutexes May 25 16:05:07.509741 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: thunder lock: enabled May 25 16:05:07.509741 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 25 16:05:07.509741 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 25 16:05:07.555225 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: Python main interpreter initialized at 0x56543467a9c0 May 25 16:05:07.555225 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: python threads support enabled May 25 16:05:07.555225 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: your server socket listen backlog is limited to 100 connections May 25 16:05:07.555225 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: your mercy for graceful operations on workers is 90 seconds May 25 16:05:07.555626 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: mapped 403077 bytes (393 KB) for 2 cores May 25 16:05:07.555700 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: *** Operational MODE: preforking *** May 25 16:05:07.555956 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: *** uWSGI is running in multiple interpreter mode *** May 25 16:05:07.555956 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: spawned uWSGI master process (pid: 80152) May 25 16:05:07.556171 dv-cinder-843278-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 25 16:05:07.557338 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: spawned uWSGI worker 1 (pid: 80167, cores: 1) May 25 16:05:07.557338 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: spawned uWSGI worker 2 (pid: 80168, cores: 1) May 25 16:05:07.557338 dv-cinder-843278-2-iscsi devstack@keystone.service[80152]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 25 16:05:13.267637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:05:13.267637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: from cryptography.utils import int_from_bytes May 25 16:05:13.268450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:05:13.268450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: from cryptography.utils import int_from_bytes May 25 16:05:13.309818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:05:13.309818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: from cryptography.utils import int_from_bytes May 25 16:05:13.310421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:05:13.310421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: from cryptography.utils import int_from_bytes May 25 16:05:13.317867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 16:05:13.318061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 16:05:13.318256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] command line args: [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 16:05:13.318529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 16:05:13.318708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ================================================================================ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 16:05:13.318962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] admin_token = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.319209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_dir = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.319395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.319576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_source = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.319757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] debug = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.319954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.321361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.321361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.321361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.321361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.321361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.321361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_config_append = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.321833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.321833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_dir = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.321968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.322111 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.322364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.322614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.322863 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.323041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.323219 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.323465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.323713 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.323949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.324190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.324426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.324661 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.324909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.325155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.325430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.325953 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.326130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] publish_errors = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.326363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.326593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.326771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.327016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.327251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.327484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] standard_threads = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.327715 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.327958 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.328639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.328900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_journal = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.329080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_json = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.329256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_stderr = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.329432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_syslog = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.329669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.329956 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.330199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.330463 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.330708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.330961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.331203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.331456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.331696 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.331942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.332197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.332451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.332688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.332937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.333173 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.333424 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.333681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.333943 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.334206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.334471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.334716 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.334954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.335210 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.335480 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.335731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.335986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.336222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.336456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.336712 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.336929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.337182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.337434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.337672 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.337929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.338181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.338433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.338677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.338931 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.339169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.339413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.339692 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.339948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.340185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.340419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.340653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.340894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.341148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.341382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.341621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.341877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.342118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.342354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.342604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.342842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.343083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.343323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.343562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.343892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.344665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.344896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.345147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.345388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.345629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.345990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.346155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.346993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.346993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.346993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.347400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.347400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.347576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.347828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.348080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.348320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.348559 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.348797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.349048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.349288 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.349527 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.349792 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.350034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.350276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.350514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.350752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.350989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.351256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.351495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.351737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.351976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.352220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.352461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.352700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.352938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.353169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.353404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.353637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.353896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.354129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.354374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.354617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.354867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.355105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.355364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.355626 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.355870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.356107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.external = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.356336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.356636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.356872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.357106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.password = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.357429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.token = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.357604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.357834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.358086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.358319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.358569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.358731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 16:05:13.358826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.358927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 16:05:13.359044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.359128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] command line args: [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 16:05:13.359281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.359393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 16:05:13.359580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.359678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ================================================================================ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 16:05:13.359832 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.360089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] admin_token = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.360265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.360385 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_dir = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.360514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.360598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.360753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_source = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.360846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.360937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] debug = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.361020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.361141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.361280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.361379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.361493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.361588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.361751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.361834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.362011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.362088 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.362278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.362367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.362448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_config_append = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.362520 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.362628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.362787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_dir = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.362892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.363004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.363110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.363218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.363312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.363407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.363531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.363647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.363745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.363894 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.363978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.364097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.364201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.364333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.364419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.364538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.364678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.364764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.364918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.365003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.365160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.365247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.365393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.365482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.365628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.365738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.365901 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.365984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.366150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.366272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.366387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.366464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.366639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.366890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.366968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.367094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] publish_errors = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.367179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.367291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.367405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.367508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.367586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.367716 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.367833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.367909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.368025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.368136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.368274 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.368403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] standard_threads = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.368546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.368670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.368753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.368826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.368978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.369065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.369144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_journal = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.369216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.369315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_json = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.369441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.369531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_stderr = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.369651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_syslog = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.369751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.369910 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 16:05:13.369988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.370166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.370244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.370398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.370475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.370652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.370734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.370891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.370975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.371140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.371220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.371374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.371450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.371614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.371693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.371848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.371933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.372087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.372165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.372332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.372410 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.372578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.372667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.372812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.372892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.373058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.373135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.373292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.373379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.373539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.373617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.373808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.373894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.374047 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.374142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.374302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.374382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.374542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.374619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.374785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.374872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.375051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.375164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.375409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.375513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.375787 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.376016 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.376096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.376261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.376351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.376503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.376583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.376755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.376866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.376989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.377074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.377222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.377302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.377454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.377539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.377703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.377861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.377977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.378056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.378210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.378289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.378444 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.378526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.378679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.378757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.378912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.379035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.379154 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.379236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.379401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.379488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.379636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.379717 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.379873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.379954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.380121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.380198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.380350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.380427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.380593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.380674 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.380822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.380899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.381053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.381130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.381286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.381365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.381523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.381602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.381785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.381869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.382026 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.382108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.382257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.382344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.382493 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.382573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.382726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.382808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.382974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.383053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.383203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.383281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.383439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.383519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.383674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.383753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.383908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.383987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.384142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.384224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.384375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.384458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.384601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.384753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.384855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.384934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.385089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.385167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.385323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.385411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.385553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.385632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.385833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.385918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.386070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.386158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.386300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.386381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.386537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.386614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.386770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.386847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.387020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.387100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.387255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.387334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.387493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.387571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.387735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.387816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.387969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.388105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.388203 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.388363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.388450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.388606 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.388685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.388861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.388946 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.389117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.389223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.389411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.389515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.389592 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.389664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.389838 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.389925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.390074 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.390153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.390308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.390387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.390543 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.390624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.391322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.391422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.391572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.391758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.391847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.392067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.392146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.392319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.392398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.392571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.392663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.392823 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.392903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.393059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.393137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.393298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.393386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.393531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.393611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.393787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.393871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.394022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.394114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.394257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.394336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.394486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.394565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.394726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.394937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.395019 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.395189 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.395275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.395429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.395506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.395686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.395768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.395942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.396024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.396180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.396260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.396411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.external = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.396493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.396641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.396721 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.396921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.396999 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.397156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.397235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.397387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.password = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.397466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.397617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.token = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.397696 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.397875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.397967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.398109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.398188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.398357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.398445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.398588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.398667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.398833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.398912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.399068 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.399158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.399301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.399379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.399535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.399620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.399772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.399858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.400018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.400096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.400254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.400332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.400490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.400577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.400732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.400811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.400965 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.401044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.401209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.401295 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.401453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.401532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.401693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.401839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.401955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.402047 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.402197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.402276 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.402433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.402513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.402680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.402768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.402942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.403022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.403176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.403254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.403421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.403508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.403665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.403743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.403916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.403994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.404164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.404254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.404397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.404485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.404646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.404726 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.404883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.404961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.405136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.405217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.405387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.405477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.405602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.405679 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.405852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.405931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.406100 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.406180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.406410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.406487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.406671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.406884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.406965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.407150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.407228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.407388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.407476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.407635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.407714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.407870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.407949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.408114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.408202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.408352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.408432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.408777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 16:05:13.408857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.409092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.409325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.409557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.409812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.410060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.410309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.410558 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.410816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.411066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.411338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.411577 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.411815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.412045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.412282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.412512 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.412746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.412983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.413220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.413467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.413698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.413974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.414220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.414453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.414698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.414946 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.415178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.415421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.415651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.415888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.416127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.416369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.416599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.416905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.417082 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.417342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.417571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.417844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.418080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.418326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.418563 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.418797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.419031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.419262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.419566 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.419849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.420131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.420378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.420626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.420873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.421107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.421336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.421571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.422453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.422622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.423227 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.423494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.423751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.424009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.424256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.424506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.424758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.424993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.425237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.425482 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.425768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.425979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.426228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.426479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.426720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.426976 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.427209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.427455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.427692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.427927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.428165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.428412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.428654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.428887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.429124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.429362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.429612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.429889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.430130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.430382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.430633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.430870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.431118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.431356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.431593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.431827 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.432075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.432305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.432554 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.432806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.433053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.433301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.433549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.433817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.434065 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.434312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.434544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.434790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.435036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.435287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.435534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.435969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.436208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.436459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.436708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.436958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.437205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.437438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.437942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.438418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.438754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.438992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.439229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.439462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.439717 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.439975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.440226 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.440466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.440695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.440929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.441162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.441399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.441639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.442029 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.442029 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.442280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.442494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.442629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.442757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.442891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.442891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.443126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.443304 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.443545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.443776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.444025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.444273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.444520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.444748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.444990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.445290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.445551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.445819 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.446070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.446317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.446572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.446775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.447012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.447239 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.447471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.447715 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.447951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.448192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.448424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.448670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.448904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.449145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.449374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.449614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.449871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 16:05:13.450041 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 16:05:13.452079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.476648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.476877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.477081 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.481688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.481948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.487353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.506755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.506938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.536127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.545172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.587762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.587993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.590282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.590518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.654780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.654963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.655055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.655157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.655238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.655346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.655430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.655566 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.655659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.655781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.655864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.655993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.679755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.679984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.680075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.680211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.680294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.680434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.680518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.680654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.680738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.680869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.680950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.681083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.681173 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.681297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.693452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.693916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.706602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.706798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.707025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.707109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.707253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.707336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.707474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.707559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.707689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.707772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.707905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.707985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.708120 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.708335 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 16:05:13.725759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.725891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.725970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 16:05:13.726063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 16:05:13.726295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.726379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.726494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 16:05:13.726575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 16:05:13.726764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.726868 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.727024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.727118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.727276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.727362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.727528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.727611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.738609 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.738892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.739225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.739441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.740617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.740891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.741161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.741442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.747547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.749511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.751459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.751774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.752020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.752301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.752581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.752838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.753217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.753498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.753664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.753799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.753968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.754078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.754247 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.754351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.754525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.754807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.755067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.755440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.755713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.755984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.756259 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.756528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.780868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.783239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.783781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.784173 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.785925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.786247 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.789037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.789329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.789600 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.791435 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.791708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.791984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.797798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.800167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.800445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.802244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.802649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.802929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.803212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.803485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.803758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.804025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.804292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.804886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.805282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.805569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.805868 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.806146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.806420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.806689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.806954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.823255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.823528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.825864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.826131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.828257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.831236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.832094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.832386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.834876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.835166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.837500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.840188 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.841097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.844346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.845510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.845839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.846133 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.846414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.846699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.846977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.848436 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.848732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.849020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.849308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.849582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.849892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.866648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.866935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.867199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.867459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.869735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.870012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.870276 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.870536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.875511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.877216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.877488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.878964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.880548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.880647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.880825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.880911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.881103 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.881383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.881656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.883959 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.884245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.884524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.884804 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.885077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.895694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.895972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.896260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.896541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.896830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.897108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.897390 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.899259 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.899536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.899818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.900087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.900361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.900629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.900906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.915805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.919081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.919386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.923029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.923516 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.924655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.926315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.926603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.926841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.926927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.928463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.930087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.930367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.930661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.934261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.935866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.938523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.940127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.940448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.940935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.941439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.941947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.943369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.943650 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.943929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.945606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.952644 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.952934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.954012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.960563 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.960853 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.961249 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.963313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.963596 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.963877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.964161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.964435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.964707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.964997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.965268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.965553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.965850 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.966123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.966383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.966653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.966918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.969876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 16:05:13.970360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.970665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.970941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.971237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.971477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.971747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.972027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.972293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.972568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.972851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.973112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.973373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:13.973637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 16:05:14.006989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56543467a9c0 pid: 80168 (default app) May 25 16:05:14.010351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b5811-d157-4218-837c-ec72a8d004d8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:14.010646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b5811-d157-4218-837c-ec72a8d004d8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:14.010924 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b5811-d157-4218-837c-ec72a8d004d8 None None] PATH_INFO: `/v3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:14.013345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56543467a9c0 pid: 80167 (default app) May 25 16:05:14.015273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Wed May 25 16:05:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:22.050539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d79c2d-d3e7-4c77-8fad-d56b03001a02 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:22.051122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d79c2d-d3e7-4c77-8fad-d56b03001a02 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:22.051122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d79c2d-d3e7-4c77-8fad-d56b03001a02 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:22.055541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:22.061056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee550dc-37e7-49d4-9842-8c1d76529796 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:22.061453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee550dc-37e7-49d4-9842-8c1d76529796 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:22.061596 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee550dc-37e7-49d4-9842-8c1d76529796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:22.131332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: INFO dbcounter [None req-dee550dc-37e7-49d4-9842-8c1d76529796 None None] Registered counter for database keystone May 25 16:05:22.153635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG oslo_db.sqlalchemy.engines [None req-dee550dc-37e7-49d4-9842-8c1d76529796 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80168) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:05:22.160491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writer thread running {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:05:22.345995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG passlib.handlers.bcrypt [None req-dee550dc-37e7-49d4-9842-8c1d76529796 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80168) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 16:05:22.346405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG passlib.handlers.bcrypt [None req-dee550dc-37e7-49d4-9842-8c1d76529796 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80168) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 16:05:22.403317 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dee550dc-37e7-49d4-9842-8c1d76529796 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:22.404952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dee550dc-37e7-49d4-9842-8c1d76529796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:22.531992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 25 16:05:22.684612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d79c2d-d3e7-4c77-8fad-d56b03001a02 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:22.751992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: INFO dbcounter [None req-d0d79c2d-d3e7-4c77-8fad-d56b03001a02 None None] Registered counter for database keystone May 25 16:05:22.771369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG oslo_db.sqlalchemy.engines [None req-d0d79c2d-d3e7-4c77-8fad-d56b03001a02 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80167) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 16:05:22.777624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writer thread running {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 16:05:22.900288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d79c2d-d3e7-4c77-8fad-d56b03001a02 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:22.904629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610e2de-750d-4a72-a432-5505fc2c1442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:22.905288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610e2de-750d-4a72-a432-5505fc2c1442 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:22.905571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610e2de-750d-4a72-a432-5505fc2c1442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:22.905875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610e2de-750d-4a72-a432-5505fc2c1442 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:22.914523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610e2de-750d-4a72-a432-5505fc2c1442 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:22.971234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c610e2de-750d-4a72-a432-5505fc2c1442 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 16:05:22.971489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610e2de-750d-4a72-a432-5505fc2c1442 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:22.981751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Wed May 25 16:05:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:05:22.988718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee550dc-37e7-49d4-9842-8c1d76529796 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:23.003970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee550dc-37e7-49d4-9842-8c1d76529796 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:23.008325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5380d1-429a-4754-ab38-4233cd861f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:23.009446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5380d1-429a-4754-ab38-4233cd861f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:23.009757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5380d1-429a-4754-ab38-4233cd861f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:23.010038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5380d1-429a-4754-ab38-4233cd861f14 admin admin] PATH_INFO: `/v3/projects/55f82ef7ffb1469ea7582069b5da4934` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:23.011529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5380d1-429a-4754-ab38-4233cd861f14 admin admin] RBAC: Authorizing `identity:get_project(project_id=55f82ef7ffb1469ea7582069b5da4934)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:23.173317 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8e5380d1-429a-4754-ab38-4233cd861f14 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 16:05:23.174038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5380d1-429a-4754-ab38-4233cd861f14 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:23.178371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Wed May 25 16:05:22 2022] GET /identity/v3/projects/55f82ef7ffb1469ea7582069b5da4934 => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:25.379546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80576c0e-b51e-45b1-a564-a76c7a1a4a06 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:25.380068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80576c0e-b51e-45b1-a564-a76c7a1a4a06 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:25.380169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80576c0e-b51e-45b1-a564-a76c7a1a4a06 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:25.384281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:25.389815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801484a-9203-4229-b57a-2f152213dd93 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:25.390259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801484a-9203-4229-b57a-2f152213dd93 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:25.390912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801484a-9203-4229-b57a-2f152213dd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:25.411060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7801484a-9203-4229-b57a-2f152213dd93 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:25.412563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7801484a-9203-4229-b57a-2f152213dd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:25.420370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 16:05:25.443123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80576c0e-b51e-45b1-a564-a76c7a1a4a06 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:25.455630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80576c0e-b51e-45b1-a564-a76c7a1a4a06 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:25.459860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971b834-0664-460a-91cf-f7b327876d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:25.460500 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b834-0664-460a-91cf-f7b327876d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:25.460788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b834-0664-460a-91cf-f7b327876d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:25.461067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b834-0664-460a-91cf-f7b327876d61 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:25.469103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971b834-0664-460a-91cf-f7b327876d61 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:25.471210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971b834-0664-460a-91cf-f7b327876d61 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:25.480186 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Wed May 25 16:05:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:05:25.486449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801484a-9203-4229-b57a-2f152213dd93 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:25.495660 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801484a-9203-4229-b57a-2f152213dd93 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:25.500699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144d95d-cde8-449d-b1b9-a086ff3de78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:25.501560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144d95d-cde8-449d-b1b9-a086ff3de78e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:25.501880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144d95d-cde8-449d-b1b9-a086ff3de78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:25.502161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144d95d-cde8-449d-b1b9-a086ff3de78e admin admin] PATH_INFO: `/v3/users/cada5de94a8d4bee93afa64a6b5b9d33` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:25.503581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144d95d-cde8-449d-b1b9-a086ff3de78e admin admin] RBAC: Authorizing `identity:get_user(user_id=cada5de94a8d4bee93afa64a6b5b9d33)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:25.505417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144d95d-cde8-449d-b1b9-a086ff3de78e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:25.507027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Wed May 25 16:05:25 2022] GET /identity/v3/users/cada5de94a8d4bee93afa64a6b5b9d33 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:27.777170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99690-1fc0-4a1b-9a1e-e990d4e0a8ae None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:27.777676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99690-1fc0-4a1b-9a1e-e990d4e0a8ae None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:27.777676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99690-1fc0-4a1b-9a1e-e990d4e0a8ae None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:27.779129 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:05:27.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0bfa-a47d-483b-b444-89253c31cc3e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:27.785882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0bfa-a47d-483b-b444-89253c31cc3e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:27.785978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0bfa-a47d-483b-b444-89253c31cc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:27.803220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d0cd0bfa-a47d-483b-b444-89253c31cc3e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:27.804519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d0cd0bfa-a47d-483b-b444-89253c31cc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:27.847884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d7711-8df7-4311-87da-b1523150acc7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:27.848178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d7711-8df7-4311-87da-b1523150acc7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:27.848456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d7711-8df7-4311-87da-b1523150acc7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:27.850049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:05:27.855290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ccf66-50f3-49ed-b3a5-aa0aa5d540b6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:27.855609 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ccf66-50f3-49ed-b3a5-aa0aa5d540b6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:27.855863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ccf66-50f3-49ed-b3a5-aa0aa5d540b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:27.887311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:27.895855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG passlib.handlers.bcrypt [None req-3f5ccf66-50f3-49ed-b3a5-aa0aa5d540b6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80167) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 16:05:27.896284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG passlib.handlers.bcrypt [None req-3f5ccf66-50f3-49ed-b3a5-aa0aa5d540b6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80167) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 16:05:27.910930 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0bfa-a47d-483b-b444-89253c31cc3e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:27.935043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:27.943926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3f5ccf66-50f3-49ed-b3a5-aa0aa5d540b6 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:27.945915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3f5ccf66-50f3-49ed-b3a5-aa0aa5d540b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:28.023040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0bfa-a47d-483b-b444-89253c31cc3e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:28.053078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:05:28.057385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd9f28-b646-4af1-88f2-8654548c97ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:28.058220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd9f28-b646-4af1-88f2-8654548c97ee None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:28.058502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd9f28-b646-4af1-88f2-8654548c97ee None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:28.058777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd9f28-b646-4af1-88f2-8654548c97ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:28.070323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd9f28-b646-4af1-88f2-8654548c97ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:28.075982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ccf66-50f3-49ed-b3a5-aa0aa5d540b6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-71dd9f28-b646-4af1-88f2-8654548c97ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:28.081580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:28.082629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:05:28.083443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:05:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:05:28.088449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd9f28-b646-4af1-88f2-8654548c97ee None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:28.172382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:28.182531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ccf66-50f3-49ed-b3a5-aa0aa5d540b6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:28.184916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:28.191536 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd9f28-b646-4af1-88f2-8654548c97ee None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:28.222314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f5557-7b3b-4993-9246-6a7e1874bc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:28.223240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f5557-7b3b-4993-9246-6a7e1874bc85 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:28.223534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f5557-7b3b-4993-9246-6a7e1874bc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:28.223817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f5557-7b3b-4993-9246-6a7e1874bc85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:28.225437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dbcef-58ee-42e8-96f8-a5614bc4d95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:28.226131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dbcef-58ee-42e8-96f8-a5614bc4d95b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:28.226416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dbcef-58ee-42e8-96f8-a5614bc4d95b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:28.226689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dbcef-58ee-42e8-96f8-a5614bc4d95b None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:28.227523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404dbcef-58ee-42e8-96f8-a5614bc4d95b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:28.227887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404dbcef-58ee-42e8-96f8-a5614bc4d95b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:28.229293 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404dbcef-58ee-42e8-96f8-a5614bc4d95b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:28.242492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:28.247872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dbcef-58ee-42e8-96f8-a5614bc4d95b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:28.303908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f5557-7b3b-4993-9246-6a7e1874bc85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:28.305381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f5557-7b3b-4993-9246-6a7e1874bc85 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:28.350155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dbcef-58ee-42e8-96f8-a5614bc4d95b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:28.385401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635383c-72fc-464f-a9e6-e47292d95cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:28.386312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635383c-72fc-464f-a9e6-e47292d95cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:28.386595 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635383c-72fc-464f-a9e6-e47292d95cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:28.386875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635383c-72fc-464f-a9e6-e47292d95cf9 None admin] PATH_INFO: `/v3/users/cada5de94a8d4bee93afa64a6b5b9d33` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:28.388104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635383c-72fc-464f-a9e6-e47292d95cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=cada5de94a8d4bee93afa64a6b5b9d33)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:28.389605 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635383c-72fc-464f-a9e6-e47292d95cf9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:28.391359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 9/16] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:28 2022] GET /identity/v3/users/cada5de94a8d4bee93afa64a6b5b9d33 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:28.396700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635383c-72fc-464f-a9e6-e47292d95cf9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:28.406247 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 8/17] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 16:05:28.411514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:28.412108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f5557-7b3b-4993-9246-6a7e1874bc85 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:28.415450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:28.509565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635383c-72fc-464f-a9e6-e47292d95cf9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:28.516219 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f5557-7b3b-4993-9246-6a7e1874bc85 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:28.543659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90074986-8e80-4b9b-8585-99308a4dc4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:28.544461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90074986-8e80-4b9b-8585-99308a4dc4ed None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:28.544745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90074986-8e80-4b9b-8585-99308a4dc4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:28.545021 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90074986-8e80-4b9b-8585-99308a4dc4ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:28.546244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90074986-8e80-4b9b-8585-99308a4dc4ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:28.548580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90074986-8e80-4b9b-8585-99308a4dc4ed None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:28.550227 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:28.551400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60903f-0e28-422a-8d13-252c05089e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:28.552210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60903f-0e28-422a-8d13-252c05089e49 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:28.552498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60903f-0e28-422a-8d13-252c05089e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:28.552778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60903f-0e28-422a-8d13-252c05089e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:28.555863 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90074986-8e80-4b9b-8585-99308a4dc4ed None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:28.572602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60903f-0e28-422a-8d13-252c05089e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:28.574131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60903f-0e28-422a-8d13-252c05089e49 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4d60903f-0e28-422a-8d13-252c05089e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:05:28.596315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:28.597180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:28.598013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:05:28.598013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:05:28.598013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:05:28.598013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:28.598013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:05:28.603521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60903f-0e28-422a-8d13-252c05089e49 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:28.641861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:28.661236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90074986-8e80-4b9b-8585-99308a4dc4ed None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:28.663162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:28.696745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3d64c-2fc0-47fa-a675-db3daf10730c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:28.697402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3d64c-2fc0-47fa-a675-db3daf10730c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:28.697685 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3d64c-2fc0-47fa-a675-db3daf10730c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:28.697992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3d64c-2fc0-47fa-a675-db3daf10730c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:28.698830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3d64c-2fc0-47fa-a675-db3daf10730c None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.domain.id=default, user.id=cada5de94a8d4bee93afa64a6b5b9d33) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:28.699138 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3d64c-2fc0-47fa-a675-db3daf10730c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:28.700598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3d64c-2fc0-47fa-a675-db3daf10730c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:28.707640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Wed May 25 16:05:28 2022] GET /identity/v3/role_assignments?user.id=cada5de94a8d4bee93afa64a6b5b9d33&scope.domain.id=default&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:28.707952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60903f-0e28-422a-8d13-252c05089e49 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:28.745850 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28cfd3-c95a-41fe-bab1-f5faf10cbcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:28.746784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28cfd3-c95a-41fe-bab1-f5faf10cbcd3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:28.746784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28cfd3-c95a-41fe-bab1-f5faf10cbcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:28.747038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28cfd3-c95a-41fe-bab1-f5faf10cbcd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:28.747791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28cfd3-c95a-41fe-bab1-f5faf10cbcd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:28.748093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28cfd3-c95a-41fe-bab1-f5faf10cbcd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:28.749546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28cfd3-c95a-41fe-bab1-f5faf10cbcd3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:28.769153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:31.185099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcb107-77df-4f2f-955b-8d18dd0b65ca None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:31.185752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcb107-77df-4f2f-955b-8d18dd0b65ca None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:31.185752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcb107-77df-4f2f-955b-8d18dd0b65ca None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:31.187221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:31.193094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71012732-b289-4aa7-bdde-db8e2bfd4c72 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:31.194027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71012732-b289-4aa7-bdde-db8e2bfd4c72 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:31.194536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71012732-b289-4aa7-bdde-db8e2bfd4c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:31.219392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-71012732-b289-4aa7-bdde-db8e2bfd4c72 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:31.220621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-71012732-b289-4aa7-bdde-db8e2bfd4c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:31.431366 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 11/23] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:31.465260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bcb107-77df-4f2f-955b-8d18dd0b65ca None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:31.538781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8db48c-dcee-4639-8705-f3915dda59e4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:31.539321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8db48c-dcee-4639-8705-f3915dda59e4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:31.539747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8db48c-dcee-4639-8705-f3915dda59e4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:31.541317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 12/24] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:05:31.547079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77d7ed-ce4b-4633-acc2-a31895b7dc8e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:31.547587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77d7ed-ce4b-4633-acc2-a31895b7dc8e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:31.548092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77d7ed-ce4b-4633-acc2-a31895b7dc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:31.571520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ab77d7ed-ce4b-4633-acc2-a31895b7dc8e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:31.572721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ab77d7ed-ce4b-4633-acc2-a31895b7dc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:31.684694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:31.692178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bcb107-77df-4f2f-955b-8d18dd0b65ca None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:31.758746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 13/25] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:31.774213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b49d33-1bb9-4ede-a424-2d2991a6f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:31.775023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b49d33-1bb9-4ede-a424-2d2991a6f6cd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:31.775312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b49d33-1bb9-4ede-a424-2d2991a6f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:31.775597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b49d33-1bb9-4ede-a424-2d2991a6f6cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:31.792849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b49d33-1bb9-4ede-a424-2d2991a6f6cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:31.801773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77d7ed-ce4b-4633-acc2-a31895b7dc8e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d6b49d33-1bb9-4ede-a424-2d2991a6f6cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:31.804736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:31.805588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:31.806376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:05:31.816106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 13/26] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:05:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:31.823231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b49d33-1bb9-4ede-a424-2d2991a6f6cd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:31.830522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:31.992242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77d7ed-ce4b-4633-acc2-a31895b7dc8e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:32.037409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b49d33-1bb9-4ede-a424-2d2991a6f6cd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:32.075926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f93c81-9706-49c3-9809-a25ec0c3434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:32.076750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f93c81-9706-49c3-9809-a25ec0c3434e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.077037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f93c81-9706-49c3-9809-a25ec0c3434e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.077316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f93c81-9706-49c3-9809-a25ec0c3434e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.083024 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e89367-0c4b-41c3-8013-05918fab396b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:32.083700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e89367-0c4b-41c3-8013-05918fab396b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.083990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e89367-0c4b-41c3-8013-05918fab396b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.084268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e89367-0c4b-41c3-8013-05918fab396b None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.085029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e89367-0c4b-41c3-8013-05918fab396b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:32.085335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e89367-0c4b-41c3-8013-05918fab396b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:32.086750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e89367-0c4b-41c3-8013-05918fab396b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:32.098439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f93c81-9706-49c3-9809-a25ec0c3434e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:32.099933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f93c81-9706-49c3-9809-a25ec0c3434e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:32.103831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 14/27] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:32.106377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9905bb7-b668-4199-9401-9c2a7b4b15cb None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.106848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9905bb7-b668-4199-9401-9c2a7b4b15cb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.107267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9905bb7-b668-4199-9401-9c2a7b4b15cb None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.108681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 15/28] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:32.110751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e13ac-3723-45e4-a310-2a352575cf46 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.111233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e13ac-3723-45e4-a310-2a352575cf46 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.111745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e13ac-3723-45e4-a310-2a352575cf46 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.113166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 16/29] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:32.117769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e13ac-3723-45e4-a310-2a352575cf46 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-47f93c81-9706-49c3-9809-a25ec0c3434e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:05:32.120477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:32.121446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:32.122372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:05:32.122372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:05:32.122372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:05:32.122372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:32.122372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 14/30] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:05:32.123255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041e40b-4639-43ad-9575-377f1e073f46 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.123551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041e40b-4639-43ad-9575-377f1e073f46 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.123835 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041e40b-4639-43ad-9575-377f1e073f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.150042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6041e40b-4639-43ad-9575-377f1e073f46 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:32.152149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6041e40b-4639-43ad-9575-377f1e073f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:32.183294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:32.199152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:32.245343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 15/31] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:32.247179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e13ac-3723-45e4-a310-2a352575cf46 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:32.247505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe17c-8881-4292-80df-fd969ca485ea None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.247942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe17c-8881-4292-80df-fd969ca485ea None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.248365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe17c-8881-4292-80df-fd969ca485ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.269967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-65ffe17c-8881-4292-80df-fd969ca485ea None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:32.271837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65ffe17c-8881-4292-80df-fd969ca485ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:32.291718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a1c3a-cc8f-4b11-b87f-5c3902f61ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:32.293229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a1c3a-cc8f-4b11-b87f-5c3902f61ced None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.294094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a1c3a-cc8f-4b11-b87f-5c3902f61ced None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.294525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a1c3a-cc8f-4b11-b87f-5c3902f61ced None admin] PATH_INFO: `/v3/users/cada5de94a8d4bee93afa64a6b5b9d33` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.295929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a1c3a-cc8f-4b11-b87f-5c3902f61ced None admin] RBAC: Authorizing `identity:get_user(user_id=cada5de94a8d4bee93afa64a6b5b9d33)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:32.298756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a1c3a-cc8f-4b11-b87f-5c3902f61ced None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:32.301420 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 17/32] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:32 2022] GET /identity/v3/users/cada5de94a8d4bee93afa64a6b5b9d33 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:32.302444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a1c3a-cc8f-4b11-b87f-5c3902f61ced None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:32.370630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 16/33] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:32.372450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffe17c-8881-4292-80df-fd969ca485ea None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:32.416287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a1c3a-cc8f-4b11-b87f-5c3902f61ced None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:32.462257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6da1ee-8386-4b8b-8626-4cd74d02e864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:32.463174 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6da1ee-8386-4b8b-8626-4cd74d02e864 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.463494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6da1ee-8386-4b8b-8626-4cd74d02e864 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.463899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6da1ee-8386-4b8b-8626-4cd74d02e864 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.465038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6da1ee-8386-4b8b-8626-4cd74d02e864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:32.465164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6da1ee-8386-4b8b-8626-4cd74d02e864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:32.466736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6da1ee-8386-4b8b-8626-4cd74d02e864 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:32.469404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:32.488394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 18/34] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:32.492896 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:32.494392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6da1ee-8386-4b8b-8626-4cd74d02e864 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:32.510796 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffe17c-8881-4292-80df-fd969ca485ea None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:32.563690 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaca453-31e7-4e99-a1b7-8943c0b1c129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:32.564662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaca453-31e7-4e99-a1b7-8943c0b1c129 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.565355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaca453-31e7-4e99-a1b7-8943c0b1c129 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.565806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaca453-31e7-4e99-a1b7-8943c0b1c129 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.567461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaca453-31e7-4e99-a1b7-8943c0b1c129 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:32.569702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaca453-31e7-4e99-a1b7-8943c0b1c129 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:32.617199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6da1ee-8386-4b8b-8626-4cd74d02e864 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:32.661773 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eab3b-1c9f-4465-aab7-c0f4a1a5f533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:32.662953 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eab3b-1c9f-4465-aab7-c0f4a1a5f533 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.663460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eab3b-1c9f-4465-aab7-c0f4a1a5f533 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.663938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eab3b-1c9f-4465-aab7-c0f4a1a5f533 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.665498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eab3b-1c9f-4465-aab7-c0f4a1a5f533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:32.667385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eab3b-1c9f-4465-aab7-c0f4a1a5f533 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:32.669393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 19/35] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:32.670431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eab3b-1c9f-4465-aab7-c0f4a1a5f533 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:32.729227 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 17/36] 10.222.0.42 () {60 vars in 1131 bytes} [Wed May 25 16:05:32 2022] POST /identity/v3/roles => generated 219 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:05:32.730557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaca453-31e7-4e99-a1b7-8943c0b1c129 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:32.746616 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:32.807039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eab3b-1c9f-4465-aab7-c0f4a1a5f533 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:32.855108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaca453-31e7-4e99-a1b7-8943c0b1c129 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:32.870604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4552f77-5b98-4a8c-ba00-8f2bc8bde132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:32.871666 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4552f77-5b98-4a8c-ba00-8f2bc8bde132 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.872175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4552f77-5b98-4a8c-ba00-8f2bc8bde132 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.872749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4552f77-5b98-4a8c-ba00-8f2bc8bde132 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.873765 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4552f77-5b98-4a8c-ba00-8f2bc8bde132 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:32.875937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4552f77-5b98-4a8c-ba00-8f2bc8bde132 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:32.901300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43affa-b575-4915-8d4f-92d85d504931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:32.902110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43affa-b575-4915-8d4f-92d85d504931 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:32.902401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43affa-b575-4915-8d4f-92d85d504931 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:32.902686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43affa-b575-4915-8d4f-92d85d504931 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:32.903756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43affa-b575-4915-8d4f-92d85d504931 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:32.905466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43affa-b575-4915-8d4f-92d85d504931 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:32.941844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:32.949255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 18/37] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:05:32 2022] POST /identity/v3/projects => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:05:32.953893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43affa-b575-4915-8d4f-92d85d504931 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:33.063020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 20/38] 10.222.0.42 () {60 vars in 1131 bytes} [Wed May 25 16:05:32 2022] POST /identity/v3/roles => generated 225 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:05:33.065212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203825b-663a-484c-9878-db6b59e60f80 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:33.065774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203825b-663a-484c-9878-db6b59e60f80 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:33.066272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203825b-663a-484c-9878-db6b59e60f80 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:33.067899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 21/39] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:33.073878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e15b43-f85c-4790-8ab0-74128f18bf3c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:33.074497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e15b43-f85c-4790-8ab0-74128f18bf3c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:33.075102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e15b43-f85c-4790-8ab0-74128f18bf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:33.083451 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43affa-b575-4915-8d4f-92d85d504931 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:33.110290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-58e15b43-f85c-4790-8ab0-74128f18bf3c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:33.112254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-58e15b43-f85c-4790-8ab0-74128f18bf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:33.131731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abae9b-d50e-47fd-859a-0afd4f2a711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:33.133351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abae9b-d50e-47fd-859a-0afd4f2a711f None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:33.133644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abae9b-d50e-47fd-859a-0afd4f2a711f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:33.133956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abae9b-d50e-47fd-859a-0afd4f2a711f None admin] PATH_INFO: `/v3/domains/default/users/cada5de94a8d4bee93afa64a6b5b9d33/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:33.137195 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abae9b-d50e-47fd-859a-0afd4f2a711f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cada5de94a8d4bee93afa64a6b5b9d33, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:33.139251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abae9b-d50e-47fd-859a-0afd4f2a711f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:33.164977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 19/40] 10.222.0.42 () {58 vars in 1361 bytes} [Wed May 25 16:05:32 2022] PUT /identity/v3/domains/default/users/cada5de94a8d4bee93afa64a6b5b9d33/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:05:33.183788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effacfde-4c63-464b-8a83-e2d9da0d4a33 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:33.184099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effacfde-4c63-464b-8a83-e2d9da0d4a33 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:33.184355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effacfde-4c63-464b-8a83-e2d9da0d4a33 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:33.186464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 20/41] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:05:33.194664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454d711-50b8-4453-b047-83cbb1d04d65 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:33.195436 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454d711-50b8-4453-b047-83cbb1d04d65 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:33.195733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454d711-50b8-4453-b047-83cbb1d04d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:33.221606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d454d711-50b8-4453-b047-83cbb1d04d65 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:33.223675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d454d711-50b8-4453-b047-83cbb1d04d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:33.245686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 22/42] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:05:33.277251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e15b43-f85c-4790-8ab0-74128f18bf3c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:33.317085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:33.341762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 21/43] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:33.370941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:33.381676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454d711-50b8-4453-b047-83cbb1d04d65 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:33.444374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e15b43-f85c-4790-8ab0-74128f18bf3c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:33.508502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b85c8-d91c-41b0-b4bd-bb0317d1d5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:33.509208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b85c8-d91c-41b0-b4bd-bb0317d1d5af None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:33.509500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b85c8-d91c-41b0-b4bd-bb0317d1d5af None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:33.509852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b85c8-d91c-41b0-b4bd-bb0317d1d5af None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:33.510626 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b85c8-d91c-41b0-b4bd-bb0317d1d5af None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:33.512275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b85c8-d91c-41b0-b4bd-bb0317d1d5af None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:33.529974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 23/44] 10.222.0.42 () {60 vars in 1131 bytes} [Wed May 25 16:05:33 2022] POST /identity/v3/roles => generated 223 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:05:33.544750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454d711-50b8-4453-b047-83cbb1d04d65 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:33.601464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2d83c-5d79-4d41-8d88-17c729a6ceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:33.602825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2d83c-5d79-4d41-8d88-17c729a6ceeb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:33.603115 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2d83c-5d79-4d41-8d88-17c729a6ceeb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:33.603638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2d83c-5d79-4d41-8d88-17c729a6ceeb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:33.607022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2d83c-5d79-4d41-8d88-17c729a6ceeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:33.608637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2d83c-5d79-4d41-8d88-17c729a6ceeb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:33.613757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 22/45] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:33.618840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b85c8-d91c-41b0-b4bd-bb0317d1d5af None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:33.756249 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b85c8-d91c-41b0-b4bd-bb0317d1d5af None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:33.772472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:33.803811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4047e-1768-41aa-b8ad-afed087bfd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:33.804542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4047e-1768-41aa-b8ad-afed087bfd5b None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:33.804845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4047e-1768-41aa-b8ad-afed087bfd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:33.805123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4047e-1768-41aa-b8ad-afed087bfd5b None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:33.805871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4047e-1768-41aa-b8ad-afed087bfd5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:33.807337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4047e-1768-41aa-b8ad-afed087bfd5b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:33.836652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:05:33 2022] POST /identity/v3/projects => generated 312 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:05:35.645792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2db5f-0dd2-4178-8aa3-ea8acd9a09c6 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:35.646298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2db5f-0dd2-4178-8aa3-ea8acd9a09c6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:35.646410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2db5f-0dd2-4178-8aa3-ea8acd9a09c6 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:35.647723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 23/47] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:35.653028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3b109-5f3d-4a8c-aa13-931e3ae254a4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:35.653478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3b109-5f3d-4a8c-aa13-931e3ae254a4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:35.653738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3b109-5f3d-4a8c-aa13-931e3ae254a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:35.670646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-38e3b109-5f3d-4a8c-aa13-931e3ae254a4 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:35.672513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-38e3b109-5f3d-4a8c-aa13-931e3ae254a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:35.743629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 25/48] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:35.766973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2db5f-0dd2-4178-8aa3-ea8acd9a09c6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:35.827575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:35.869704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2db5f-0dd2-4178-8aa3-ea8acd9a09c6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:35.906733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8470802-e504-4ff8-af2a-c05706f3380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:35.907631 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470802-e504-4ff8-af2a-c05706f3380f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:35.907923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470802-e504-4ff8-af2a-c05706f3380f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:35.908206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470802-e504-4ff8-af2a-c05706f3380f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:35.926714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8470802-e504-4ff8-af2a-c05706f3380f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d8470802-e504-4ff8-af2a-c05706f3380f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:35.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:35.936720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:35.938106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 24/49] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:05:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:35.944631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3b109-5f3d-4a8c-aa13-931e3ae254a4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:36.038967 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:36.047144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3b109-5f3d-4a8c-aa13-931e3ae254a4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:36.080825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958bba2-f729-46d2-b59e-0ece01df3176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:36.081519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958bba2-f729-46d2-b59e-0ece01df3176 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:36.081838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958bba2-f729-46d2-b59e-0ece01df3176 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:36.082125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958bba2-f729-46d2-b59e-0ece01df3176 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:36.082878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958bba2-f729-46d2-b59e-0ece01df3176 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:36.083195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958bba2-f729-46d2-b59e-0ece01df3176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:36.084590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958bba2-f729-46d2-b59e-0ece01df3176 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:36.092807 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 26/50] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:36.095597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbc374-6177-4486-80cd-1f57ac08352c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:36.095895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbc374-6177-4486-80cd-1f57ac08352c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:36.096233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbc374-6177-4486-80cd-1f57ac08352c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:36.097617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 25/51] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:36.099005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958bba2-f729-46d2-b59e-0ece01df3176 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:36.102733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb92ce-21d1-47bc-80bd-a99c46586cc2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:36.103027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb92ce-21d1-47bc-80bd-a99c46586cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:36.103312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb92ce-21d1-47bc-80bd-a99c46586cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:36.120660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-53cb92ce-21d1-47bc-80bd-a99c46586cc2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:36.121924 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-53cb92ce-21d1-47bc-80bd-a99c46586cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:36.195251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 26/52] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:05:36.201365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958bba2-f729-46d2-b59e-0ece01df3176 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:36.217588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb92ce-21d1-47bc-80bd-a99c46586cc2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:36.236133 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122bba4-8d80-49a8-a0d3-4d5fa85f3a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:36.237004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122bba4-8d80-49a8-a0d3-4d5fa85f3a92 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:36.237287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122bba4-8d80-49a8-a0d3-4d5fa85f3a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:36.237563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122bba4-8d80-49a8-a0d3-4d5fa85f3a92 None admin] PATH_INFO: `/v3/users/cada5de94a8d4bee93afa64a6b5b9d33` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:36.238822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122bba4-8d80-49a8-a0d3-4d5fa85f3a92 None admin] RBAC: Authorizing `identity:get_user(user_id=cada5de94a8d4bee93afa64a6b5b9d33)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:36.240292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122bba4-8d80-49a8-a0d3-4d5fa85f3a92 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:36.241961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 27/53] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:36 2022] GET /identity/v3/users/cada5de94a8d4bee93afa64a6b5b9d33 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:36.247221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122bba4-8d80-49a8-a0d3-4d5fa85f3a92 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:36.267523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:36.273124 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:36.320191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb92ce-21d1-47bc-80bd-a99c46586cc2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:36.350646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122bba4-8d80-49a8-a0d3-4d5fa85f3a92 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:36.354175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597e648-da04-4297-9c6e-3410f282f81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:36.354979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597e648-da04-4297-9c6e-3410f282f81e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:36.355269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597e648-da04-4297-9c6e-3410f282f81e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:36.355550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597e648-da04-4297-9c6e-3410f282f81e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:36.356741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597e648-da04-4297-9c6e-3410f282f81e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:36.358245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597e648-da04-4297-9c6e-3410f282f81e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:36.360437 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 27/54] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:36.365874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597e648-da04-4297-9c6e-3410f282f81e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:36.388264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f550b7-d752-4301-bc4c-d0ccd2c5f070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:36.389070 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f550b7-d752-4301-bc4c-d0ccd2c5f070 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:36.389435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f550b7-d752-4301-bc4c-d0ccd2c5f070 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:36.389740 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f550b7-d752-4301-bc4c-d0ccd2c5f070 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:36.391606 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f550b7-d752-4301-bc4c-d0ccd2c5f070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:36.393086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f550b7-d752-4301-bc4c-d0ccd2c5f070 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:36.394797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 28/55] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:36.400462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f550b7-d752-4301-bc4c-d0ccd2c5f070 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:36.468102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597e648-da04-4297-9c6e-3410f282f81e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:36.485105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:36.495326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:36.504072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e6487-3a9b-4398-88c0-8f415305df23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:36.504502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f550b7-d752-4301-bc4c-d0ccd2c5f070 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:36.504754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e6487-3a9b-4398-88c0-8f415305df23 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:36.505041 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e6487-3a9b-4398-88c0-8f415305df23 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:36.505320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e6487-3a9b-4398-88c0-8f415305df23 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:36.506075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e6487-3a9b-4398-88c0-8f415305df23 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:36.507567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e6487-3a9b-4398-88c0-8f415305df23 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:36.534759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 28/56] 10.222.0.42 () {60 vars in 1140 bytes} [Wed May 25 16:05:36 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:05:36.545401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2126b8a-9a47-4790-913c-7421287c43ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:36.546113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2126b8a-9a47-4790-913c-7421287c43ac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:36.546402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2126b8a-9a47-4790-913c-7421287c43ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:36.546754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2126b8a-9a47-4790-913c-7421287c43ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:36.548005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2126b8a-9a47-4790-913c-7421287c43ac None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.domain.id=default, user.id=cada5de94a8d4bee93afa64a6b5b9d33) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:36.548322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2126b8a-9a47-4790-913c-7421287c43ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:36.549868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2126b8a-9a47-4790-913c-7421287c43ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:36.556682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 29/57] 10.222.0.42 () {56 vars in 1322 bytes} [Wed May 25 16:05:36 2022] GET /identity/v3/role_assignments?user.id=cada5de94a8d4bee93afa64a6b5b9d33&scope.domain.id=default&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:38.815974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d7b3-aabc-4e55-879f-818ba83b866b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:38.816473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d7b3-aabc-4e55-879f-818ba83b866b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:38.816555 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d7b3-aabc-4e55-879f-818ba83b866b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:38.817977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 29/58] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:38.823148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61de927-0203-41c0-83e4-2a9b4df23ba2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:38.823565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61de927-0203-41c0-83e4-2a9b4df23ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:38.823710 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61de927-0203-41c0-83e4-2a9b4df23ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:38.841431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b61de927-0203-41c0-83e4-2a9b4df23ba2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:38.842625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b61de927-0203-41c0-83e4-2a9b4df23ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:38.907595 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 30/59] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:38.932409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572d7b3-aabc-4e55-879f-818ba83b866b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:39.035373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572d7b3-aabc-4e55-879f-818ba83b866b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:39.069730 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2ed7c-74f0-441f-a9bd-be420ab648c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:39.070466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2ed7c-74f0-441f-a9bd-be420ab648c2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:39.070754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2ed7c-74f0-441f-a9bd-be420ab648c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:39.071035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2ed7c-74f0-441f-a9bd-be420ab648c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:39.072221 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2ed7c-74f0-441f-a9bd-be420ab648c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:39.073687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2ed7c-74f0-441f-a9bd-be420ab648c2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:39.075376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 30/60] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:39.081126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61de927-0203-41c0-83e4-2a9b4df23ba2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:39.164800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:39.181523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61de927-0203-41c0-83e4-2a9b4df23ba2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:39.215424 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad211d-6d99-49bf-88d9-2689b99c1e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:39.216106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad211d-6d99-49bf-88d9-2689b99c1e0a None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:39.216392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad211d-6d99-49bf-88d9-2689b99c1e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:39.216670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad211d-6d99-49bf-88d9-2689b99c1e0a None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:39.217379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad211d-6d99-49bf-88d9-2689b99c1e0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:39.219575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad211d-6d99-49bf-88d9-2689b99c1e0a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:39.246136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 31/61] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:05:39 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:05:42.506855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcb2cc-927c-49fd-8a4f-d8d84f549faf None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:42.507359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcb2cc-927c-49fd-8a4f-d8d84f549faf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:42.507430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcb2cc-927c-49fd-8a4f-d8d84f549faf None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:42.508813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 31/62] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:42.514429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee1cb4-8cf2-4e64-9d55-0d3b5642353c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:42.515375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee1cb4-8cf2-4e64-9d55-0d3b5642353c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:42.515552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee1cb4-8cf2-4e64-9d55-0d3b5642353c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:42.538682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ddee1cb4-8cf2-4e64-9d55-0d3b5642353c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:42.539885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ddee1cb4-8cf2-4e64-9d55-0d3b5642353c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:42.632844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 32/63] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:42.671814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcb2cc-927c-49fd-8a4f-d8d84f549faf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:42.723655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:42.787647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2fa18-9082-4050-8b55-52da75e4cd02 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:42.787959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2fa18-9082-4050-8b55-52da75e4cd02 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:42.788236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2fa18-9082-4050-8b55-52da75e4cd02 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:42.789794 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 33/64] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:42.794952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5facc-1103-43e5-bff2-84e333ac05c3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:42.795250 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5facc-1103-43e5-bff2-84e333ac05c3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:42.795535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5facc-1103-43e5-bff2-84e333ac05c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:42.815750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c0b5facc-1103-43e5-bff2-84e333ac05c3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:42.816961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c0b5facc-1103-43e5-bff2-84e333ac05c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:42.830883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcb2cc-927c-49fd-8a4f-d8d84f549faf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:42.907939 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 34/65] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:42.909068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f55e38-1007-48b0-98f8-8f231db8b881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:42.910183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f55e38-1007-48b0-98f8-8f231db8b881 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:42.910615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f55e38-1007-48b0-98f8-8f231db8b881 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:42.912106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429c30c-b636-4aa6-bd16-3676997cf5a7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:42.912536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f55e38-1007-48b0-98f8-8f231db8b881 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:42.913476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429c30c-b636-4aa6-bd16-3676997cf5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:42.915600 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429c30c-b636-4aa6-bd16-3676997cf5a7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:42.917139 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 35/66] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:42.927102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a04fd8-9f28-4c35-91bf-27069ca91c41 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:42.927584 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a04fd8-9f28-4c35-91bf-27069ca91c41 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:42.928011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a04fd8-9f28-4c35-91bf-27069ca91c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:42.938306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f55e38-1007-48b0-98f8-8f231db8b881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:42.939168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-33f55e38-1007-48b0-98f8-8f231db8b881 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:42.948476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:42.949391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:42.950315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:42.950886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 32/67] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:42.951886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f55e38-1007-48b0-98f8-8f231db8b881 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:42.955705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-82a04fd8-9f28-4c35-91bf-27069ca91c41 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:42.957096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-82a04fd8-9f28-4c35-91bf-27069ca91c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:43.029552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 36/68] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:43.037733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a04fd8-9f28-4c35-91bf-27069ca91c41 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:43.059496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:43.067086 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f55e38-1007-48b0-98f8-8f231db8b881 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:43.105439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fba79-37a3-4351-8d84-75757d6b1892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:43.106532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fba79-37a3-4351-8d84-75757d6b1892 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.107144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fba79-37a3-4351-8d84-75757d6b1892 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.107251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fba79-37a3-4351-8d84-75757d6b1892 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.121409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993fba79-37a3-4351-8d84-75757d6b1892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-993fba79-37a3-4351-8d84-75757d6b1892 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:43.130523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:43.131443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:43.132245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 33/69] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:05:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:43.132972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fba79-37a3-4351-8d84-75757d6b1892 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:43.173167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a04fd8-9f28-4c35-91bf-27069ca91c41 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:43.207666 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:43.213520 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e276b-7444-43a8-b195-9d3b548c7adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:43.214537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e276b-7444-43a8-b195-9d3b548c7adb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.215021 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e276b-7444-43a8-b195-9d3b548c7adb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.215495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e276b-7444-43a8-b195-9d3b548c7adb None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.216450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e276b-7444-43a8-b195-9d3b548c7adb None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:43.216974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e276b-7444-43a8-b195-9d3b548c7adb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:43.219196 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e276b-7444-43a8-b195-9d3b548c7adb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:43.228301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 37/70] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:05:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:43.229411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e276b-7444-43a8-b195-9d3b548c7adb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:43.257441 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fba79-37a3-4351-8d84-75757d6b1892 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:43.295457 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a25007-3027-410f-a019-6e5937e91887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:43.296685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25007-3027-410f-a019-6e5937e91887 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.297093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25007-3027-410f-a019-6e5937e91887 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.297597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25007-3027-410f-a019-6e5937e91887 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.309052 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a25007-3027-410f-a019-6e5937e91887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-99a25007-3027-410f-a019-6e5937e91887 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:43.317798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:43.318676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:43.319477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 34/71] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:05:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:43.321010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a25007-3027-410f-a019-6e5937e91887 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:43.348045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:43.354600 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e276b-7444-43a8-b195-9d3b548c7adb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:43.393847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93105445-0ff0-476d-be15-8c6f286fe3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:43.394525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93105445-0ff0-476d-be15-8c6f286fe3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.394811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93105445-0ff0-476d-be15-8c6f286fe3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.395093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93105445-0ff0-476d-be15-8c6f286fe3b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.395849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93105445-0ff0-476d-be15-8c6f286fe3b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:43.396171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93105445-0ff0-476d-be15-8c6f286fe3b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:43.397566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93105445-0ff0-476d-be15-8c6f286fe3b4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:43.405479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 38/72] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:05:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:43.406750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22414a2-8873-488d-b8b4-e274ace9fc67 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.407045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22414a2-8873-488d-b8b4-e274ace9fc67 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.407325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22414a2-8873-488d-b8b4-e274ace9fc67 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.409061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 39/73] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:43.409570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22414a2-8873-488d-b8b4-e274ace9fc67 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:43.441174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a25007-3027-410f-a019-6e5937e91887 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:43.471011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:43.476709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31144648-1548-4ae2-b768-8dde2312e9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:43.477605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31144648-1548-4ae2-b768-8dde2312e9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.477918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31144648-1548-4ae2-b768-8dde2312e9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.478203 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31144648-1548-4ae2-b768-8dde2312e9d3 None admin] PATH_INFO: `/v3/users/ab37fe904ac74e65a450dec01100babf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.494451 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31144648-1548-4ae2-b768-8dde2312e9d3 None admin] RBAC: Authorizing `identity:get_user(user_id=ab37fe904ac74e65a450dec01100babf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:43.495927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31144648-1548-4ae2-b768-8dde2312e9d3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:43.497655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 35/74] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:43 2022] GET /identity/v3/users/ab37fe904ac74e65a450dec01100babf => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:43.498939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e4502-b970-400b-959a-67b0ea475bba None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.499874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e4502-b970-400b-959a-67b0ea475bba None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.500055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e4502-b970-400b-959a-67b0ea475bba None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.503650 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 36/75] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:43.503650 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e4502-b970-400b-959a-67b0ea475bba None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:43.529030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22414a2-8873-488d-b8b4-e274ace9fc67 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:43.564893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6bb97-3667-4ea6-9828-e787bf73112d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:43.565573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6bb97-3667-4ea6-9828-e787bf73112d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.565897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6bb97-3667-4ea6-9828-e787bf73112d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.566180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6bb97-3667-4ea6-9828-e787bf73112d None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.566942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6bb97-3667-4ea6-9828-e787bf73112d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:43.567253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6bb97-3667-4ea6-9828-e787bf73112d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:43.568654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6bb97-3667-4ea6-9828-e787bf73112d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:43.577507 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 40/76] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:43.578807 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280bef1-1455-4716-a7ff-36453c896712 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.579222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280bef1-1455-4716-a7ff-36453c896712 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.579478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280bef1-1455-4716-a7ff-36453c896712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.585617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:43.596781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6280bef1-1455-4716-a7ff-36453c896712 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:43.598005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6280bef1-1455-4716-a7ff-36453c896712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:43.614085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e4502-b970-400b-959a-67b0ea475bba None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:43.648256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0ac38-c321-4adf-89c8-cb024340f7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:43.649140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ac38-c321-4adf-89c8-cb024340f7bb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.649428 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ac38-c321-4adf-89c8-cb024340f7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.649709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ac38-c321-4adf-89c8-cb024340f7bb None admin] PATH_INFO: `/v3/users/ab37fe904ac74e65a450dec01100babf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.650938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0ac38-c321-4adf-89c8-cb024340f7bb None admin] RBAC: Authorizing `identity:get_user(user_id=ab37fe904ac74e65a450dec01100babf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:43.652397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0ac38-c321-4adf-89c8-cb024340f7bb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:43.654086 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 37/77] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:43 2022] GET /identity/v3/users/ab37fe904ac74e65a450dec01100babf => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:43.654861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0ac38-c321-4adf-89c8-cb024340f7bb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:43.665002 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 41/78] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:43.666332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b249bdc-eda3-47e5-bf7b-43d93baa6364 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.666632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b249bdc-eda3-47e5-bf7b-43d93baa6364 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.666919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b249bdc-eda3-47e5-bf7b-43d93baa6364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.684064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8b249bdc-eda3-47e5-bf7b-43d93baa6364 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:43.685433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8b249bdc-eda3-47e5-bf7b-43d93baa6364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:43.731047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:43.754556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 42/79] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:43.755309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b249bdc-eda3-47e5-bf7b-43d93baa6364 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:43.762564 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0ac38-c321-4adf-89c8-cb024340f7bb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:43.801559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37f89f-ae6d-47fc-aa70-14a8331fcada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:43.802264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37f89f-ae6d-47fc-aa70-14a8331fcada None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.802550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37f89f-ae6d-47fc-aa70-14a8331fcada None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.802834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37f89f-ae6d-47fc-aa70-14a8331fcada None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.815531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:43.871463 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b249bdc-eda3-47e5-bf7b-43d93baa6364 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:43.871664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37f89f-ae6d-47fc-aa70-14a8331fcada None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:43.873022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37f89f-ae6d-47fc-aa70-14a8331fcada None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:43.906084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51949f8-dc1a-4045-8322-fb2966a5046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:43.906975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51949f8-dc1a-4045-8322-fb2966a5046c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.907261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51949f8-dc1a-4045-8322-fb2966a5046c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.907541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51949f8-dc1a-4045-8322-fb2966a5046c None admin] PATH_INFO: `/v3/users/cada5de94a8d4bee93afa64a6b5b9d33` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.909016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51949f8-dc1a-4045-8322-fb2966a5046c None admin] RBAC: Authorizing `identity:get_user(user_id=cada5de94a8d4bee93afa64a6b5b9d33)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:43.910514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51949f8-dc1a-4045-8322-fb2966a5046c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:43.912159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 43/80] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:43 2022] GET /identity/v3/users/cada5de94a8d4bee93afa64a6b5b9d33 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:43.916859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51949f8-dc1a-4045-8322-fb2966a5046c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:43.963562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:43.978987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 38/81] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:05:43.979889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37f89f-ae6d-47fc-aa70-14a8331fcada None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:44.017971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51949f8-dc1a-4045-8322-fb2966a5046c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:44.025467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:44.053013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fe72a-3d3c-48ba-87cf-8d3e59718107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:44.053661 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe72a-3d3c-48ba-87cf-8d3e59718107 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.053972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe72a-3d3c-48ba-87cf-8d3e59718107 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.054589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe72a-3d3c-48ba-87cf-8d3e59718107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.085475 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37f89f-ae6d-47fc-aa70-14a8331fcada None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:44.119336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd844f-8c58-4ab7-b5ba-95bd0294a6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:44.120142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd844f-8c58-4ab7-b5ba-95bd0294a6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.120433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd844f-8c58-4ab7-b5ba-95bd0294a6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.120716 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd844f-8c58-4ab7-b5ba-95bd0294a6d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.121937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd844f-8c58-4ab7-b5ba-95bd0294a6d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:44.122351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fe72a-3d3c-48ba-87cf-8d3e59718107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:44.123399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd844f-8c58-4ab7-b5ba-95bd0294a6d1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:44.123811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fe72a-3d3c-48ba-87cf-8d3e59718107 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:44.125128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 39/82] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:44.126578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd844f-8c58-4ab7-b5ba-95bd0294a6d1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:44.172476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:44.232774 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd844f-8c58-4ab7-b5ba-95bd0294a6d1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:44.236850 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 44/83] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:05:44.239973 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fe72a-3d3c-48ba-87cf-8d3e59718107 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:44.245210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:44.273074 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80d414-7fdc-4678-8735-5db04dbab044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:44.274454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80d414-7fdc-4678-8735-5db04dbab044 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.274932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80d414-7fdc-4678-8735-5db04dbab044 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.275410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80d414-7fdc-4678-8735-5db04dbab044 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.288315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80d414-7fdc-4678-8735-5db04dbab044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5d80d414-7fdc-4678-8735-5db04dbab044 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:44.299548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:44.300567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:44.301480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 40/84] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:44.302373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80d414-7fdc-4678-8735-5db04dbab044 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:44.340715 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fe72a-3d3c-48ba-87cf-8d3e59718107 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:44.374819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eb335-b568-4323-b695-ac77ca0dcfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:44.375643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eb335-b568-4323-b695-ac77ca0dcfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.375841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eb335-b568-4323-b695-ac77ca0dcfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.376120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eb335-b568-4323-b695-ac77ca0dcfc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.393155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:44.408144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80d414-7fdc-4678-8735-5db04dbab044 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:44.444391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de9d06-51ef-4a50-b1e3-a5f5899417bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:44.445267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de9d06-51ef-4a50-b1e3-a5f5899417bb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.445560 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de9d06-51ef-4a50-b1e3-a5f5899417bb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.445876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de9d06-51ef-4a50-b1e3-a5f5899417bb None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.447075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de9d06-51ef-4a50-b1e3-a5f5899417bb None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:44.448571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de9d06-51ef-4a50-b1e3-a5f5899417bb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:44.449771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35eb335-b568-4323-b695-ac77ca0dcfc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:44.450315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 41/85] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:44 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:44.451077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de9d06-51ef-4a50-b1e3-a5f5899417bb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:44.451236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35eb335-b568-4323-b695-ac77ca0dcfc9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:44.494752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:44.553938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 45/86] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:05:44.555801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eb335-b568-4323-b695-ac77ca0dcfc9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:44.556649 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de9d06-51ef-4a50-b1e3-a5f5899417bb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:44.591979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2263761-58c5-460b-a445-b0a93c52d7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:44.592604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2263761-58c5-460b-a445-b0a93c52d7ab None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.592899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2263761-58c5-460b-a445-b0a93c52d7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.593254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2263761-58c5-460b-a445-b0a93c52d7ab None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.594344 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2263761-58c5-460b-a445-b0a93c52d7ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:44.595808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2263761-58c5-460b-a445-b0a93c52d7ab None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:44.604808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:44.623693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 42/87] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:05:44 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:05:44.625824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2263761-58c5-460b-a445-b0a93c52d7ab None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:44.679675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eb335-b568-4323-b695-ac77ca0dcfc9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:44.719666 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92429895-afa2-450b-ad1c-3ea9da03a834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:44.720561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92429895-afa2-450b-ad1c-3ea9da03a834 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.720846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92429895-afa2-450b-ad1c-3ea9da03a834 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.721136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92429895-afa2-450b-ad1c-3ea9da03a834 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.722725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92429895-afa2-450b-ad1c-3ea9da03a834 None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:44.724250 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92429895-afa2-450b-ad1c-3ea9da03a834 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:44.726433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 46/88] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:44 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:44.727167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92429895-afa2-450b-ad1c-3ea9da03a834 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:44.745320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:44.753551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2263761-58c5-460b-a445-b0a93c52d7ab None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:44.798014 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b3fd1-69f6-4f20-a2fd-3ee055653441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:44.798715 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b3fd1-69f6-4f20-a2fd-3ee055653441 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.799003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b3fd1-69f6-4f20-a2fd-3ee055653441 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.799310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b3fd1-69f6-4f20-a2fd-3ee055653441 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.800030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b3fd1-69f6-4f20-a2fd-3ee055653441 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:44.800492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b3fd1-69f6-4f20-a2fd-3ee055653441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:44.801759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b3fd1-69f6-4f20-a2fd-3ee055653441 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:44.816056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 43/89] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:05:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:44.816904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b3fd1-69f6-4f20-a2fd-3ee055653441 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:44.856437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92429895-afa2-450b-ad1c-3ea9da03a834 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:44.875948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:44.901832 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbb023-7fba-4d62-823b-49eb1e3c9f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:44.902494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbb023-7fba-4d62-823b-49eb1e3c9f9b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.902780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbb023-7fba-4d62-823b-49eb1e3c9f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.903063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbb023-7fba-4d62-823b-49eb1e3c9f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.904036 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbb023-7fba-4d62-823b-49eb1e3c9f9b None admin] RBAC: Adding query filter params (role.id=7f6dd280cf8941dcb8f79ecf88dbda78, scope.project.id=82df840b0bc44187a1a9db5c2a668847, user.id=ab37fe904ac74e65a450dec01100babf) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:44.904255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbb023-7fba-4d62-823b-49eb1e3c9f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:44.906597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbb023-7fba-4d62-823b-49eb1e3c9f9b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:44.914390 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 47/90] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:05:44 2022] GET /identity/v3/role_assignments?user.id=ab37fe904ac74e65a450dec01100babf&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:44.916421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbb023-7fba-4d62-823b-49eb1e3c9f9b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:44.965977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b3fd1-69f6-4f20-a2fd-3ee055653441 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:45.015246 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae2c92-aa9b-4d63-b1da-db628c84fbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:45.016476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae2c92-aa9b-4d63-b1da-db628c84fbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:45.016773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae2c92-aa9b-4d63-b1da-db628c84fbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:45.017059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae2c92-aa9b-4d63-b1da-db628c84fbb8 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:45.019746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae2c92-aa9b-4d63-b1da-db628c84fbb8 None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:45.021271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae2c92-aa9b-4d63-b1da-db628c84fbb8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:45.023334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 44/91] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:44 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:45.023972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae2c92-aa9b-4d63-b1da-db628c84fbb8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:45.062571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:45.065216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbb023-7fba-4d62-823b-49eb1e3c9f9b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:45.124418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bf956-159f-4a45-83a9-3754e69d7579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:45.125313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bf956-159f-4a45-83a9-3754e69d7579 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:45.125609 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bf956-159f-4a45-83a9-3754e69d7579 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:45.126009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bf956-159f-4a45-83a9-3754e69d7579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:45.127063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bf956-159f-4a45-83a9-3754e69d7579 None admin] RBAC: Adding query filter params (role.id=e7180b44fc2945568dfb83dfaf8b5c79, scope.project.id=82df840b0bc44187a1a9db5c2a668847, user.id=ab37fe904ac74e65a450dec01100babf) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:45.128819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bf956-159f-4a45-83a9-3754e69d7579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:45.130407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bf956-159f-4a45-83a9-3754e69d7579 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:45.139407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 48/92] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:05:44 2022] GET /identity/v3/role_assignments?user.id=ab37fe904ac74e65a450dec01100babf&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=e7180b44fc2945568dfb83dfaf8b5c79 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:45.140610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bf956-159f-4a45-83a9-3754e69d7579 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:45.180969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae2c92-aa9b-4d63-b1da-db628c84fbb8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:45.213656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:45.237518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5c203-7874-45bc-965e-ece179f64914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:45.238465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5c203-7874-45bc-965e-ece179f64914 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:45.238755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5c203-7874-45bc-965e-ece179f64914 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:45.239037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5c203-7874-45bc-965e-ece179f64914 None admin] PATH_INFO: `/v3/users/ab37fe904ac74e65a450dec01100babf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:45.240338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b5c203-7874-45bc-965e-ece179f64914 None admin] RBAC: Authorizing `identity:get_user(user_id=ab37fe904ac74e65a450dec01100babf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:45.241880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b5c203-7874-45bc-965e-ece179f64914 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:45.244006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 45/93] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:45 2022] GET /identity/v3/users/ab37fe904ac74e65a450dec01100babf => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:45.251018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5c203-7874-45bc-965e-ece179f64914 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:45.306890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bf956-159f-4a45-83a9-3754e69d7579 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:45.371446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f0e20-2cbc-4a95-9654-f01be4130fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:45.372518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f0e20-2cbc-4a95-9654-f01be4130fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:45.372989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f0e20-2cbc-4a95-9654-f01be4130fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:45.373425 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f0e20-2cbc-4a95-9654-f01be4130fe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:45.374913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f0e20-2cbc-4a95-9654-f01be4130fe7 None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=82df840b0bc44187a1a9db5c2a668847, user.id=cada5de94a8d4bee93afa64a6b5b9d33) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:45.375395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f0e20-2cbc-4a95-9654-f01be4130fe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:45.377472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f0e20-2cbc-4a95-9654-f01be4130fe7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:45.391736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 49/94] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:05:45 2022] GET /identity/v3/role_assignments?user.id=cada5de94a8d4bee93afa64a6b5b9d33&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:45.462132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:45.474838 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5c203-7874-45bc-965e-ece179f64914 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:45.515569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0b8e9-b73a-4992-8d82-f6bedfae507e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:45.516626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0b8e9-b73a-4992-8d82-f6bedfae507e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:45.517108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0b8e9-b73a-4992-8d82-f6bedfae507e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:45.517581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0b8e9-b73a-4992-8d82-f6bedfae507e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:45.602483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0b8e9-b73a-4992-8d82-f6bedfae507e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:45.604825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0b8e9-b73a-4992-8d82-f6bedfae507e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:45.707189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:45.717986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 46/95] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:05:45.726925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f0e20-2cbc-4a95-9654-f01be4130fe7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:45.866789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f0e20-2cbc-4a95-9654-f01be4130fe7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:45.874192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:45.917371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aacd7c-10d0-44bd-83a5-2cc79cd0c718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:45.918299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aacd7c-10d0-44bd-83a5-2cc79cd0c718 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:45.918591 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aacd7c-10d0-44bd-83a5-2cc79cd0c718 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:45.918872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aacd7c-10d0-44bd-83a5-2cc79cd0c718 None admin] PATH_INFO: `/v3/projects/dcb2341a27ec40efbf28022f6b98ce9a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:45.920123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aacd7c-10d0-44bd-83a5-2cc79cd0c718 None admin] RBAC: Authorizing `identity:get_project(project_id=dcb2341a27ec40efbf28022f6b98ce9a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:45.922516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aacd7c-10d0-44bd-83a5-2cc79cd0c718 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:45.924288 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 50/96] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:45 2022] GET /identity/v3/projects/dcb2341a27ec40efbf28022f6b98ce9a => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:45.930269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0b8e9-b73a-4992-8d82-f6bedfae507e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:46.046312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0b8e9-b73a-4992-8d82-f6bedfae507e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:46.084447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9decbd-ac63-4c00-9b44-ad4931b8fcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:46.085481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9decbd-ac63-4c00-9b44-ad4931b8fcab None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:46.086015 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9decbd-ac63-4c00-9b44-ad4931b8fcab None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:46.086488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9decbd-ac63-4c00-9b44-ad4931b8fcab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:46.087575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9decbd-ac63-4c00-9b44-ad4931b8fcab None admin] RBAC: Adding query filter params (role.id=7f6dd280cf8941dcb8f79ecf88dbda78, scope.project.id=dcb2341a27ec40efbf28022f6b98ce9a, user.id=ab37fe904ac74e65a450dec01100babf) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:46.088111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9decbd-ac63-4c00-9b44-ad4931b8fcab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:46.089869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9decbd-ac63-4c00-9b44-ad4931b8fcab None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:46.098508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 47/97] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:05:45 2022] GET /identity/v3/role_assignments?user.id=ab37fe904ac74e65a450dec01100babf&scope.project.id=dcb2341a27ec40efbf28022f6b98ce9a&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:47.843475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d284-ecf4-4b2b-b952-a9929c9b0e29 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:47.844570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d284-ecf4-4b2b-b952-a9929c9b0e29 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:47.845077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d284-ecf4-4b2b-b952-a9929c9b0e29 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:47.846721 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 51/98] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:47.852582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655d9a5-e5a2-4978-9737-bbec6c791286 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:47.853453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655d9a5-e5a2-4978-9737-bbec6c791286 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:47.853977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655d9a5-e5a2-4978-9737-bbec6c791286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:47.878919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d655d9a5-e5a2-4978-9737-bbec6c791286 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:47.880506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d655d9a5-e5a2-4978-9737-bbec6c791286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:47.954255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:47.984167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 48/99] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:48.025120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888d284-ecf4-4b2b-b952-a9929c9b0e29 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:48.161032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888d284-ecf4-4b2b-b952-a9929c9b0e29 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:48.200865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fdf1f-bb8a-43c2-94b2-609466964f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:48.201736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fdf1f-bb8a-43c2-94b2-609466964f7e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:48.202062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fdf1f-bb8a-43c2-94b2-609466964f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:48.202390 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fdf1f-bb8a-43c2-94b2-609466964f7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:48.220187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fdf1f-bb8a-43c2-94b2-609466964f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0e8fdf1f-bb8a-43c2-94b2-609466964f7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:48.228431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:48.229414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:05:48.230254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 52/100] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:48.238855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655d9a5-e5a2-4978-9737-bbec6c791286 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:48.367784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655d9a5-e5a2-4978-9737-bbec6c791286 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:48.419098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba8d20-ef30-492c-9e6d-ce30ab31ef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:48.419705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba8d20-ef30-492c-9e6d-ce30ab31ef50 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:48.419993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba8d20-ef30-492c-9e6d-ce30ab31ef50 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:48.420276 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba8d20-ef30-492c-9e6d-ce30ab31ef50 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:48.421917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba8d20-ef30-492c-9e6d-ce30ab31ef50 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:48.422245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba8d20-ef30-492c-9e6d-ce30ab31ef50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:48.423654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba8d20-ef30-492c-9e6d-ce30ab31ef50 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:48.447570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 49/101] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:05:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:48.453654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fdf1f-bb8a-43c2-94b2-609466964f7e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:48.465148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe4be7-020e-450d-bcef-c57c99fd385f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:48.465458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe4be7-020e-450d-bcef-c57c99fd385f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:48.467624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe4be7-020e-450d-bcef-c57c99fd385f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:48.469944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 50/102] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:48.482356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5043-93dc-4957-82d1-78dac65decf0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:48.482658 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5043-93dc-4957-82d1-78dac65decf0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:48.482919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5043-93dc-4957-82d1-78dac65decf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:48.510306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-edcf5043-93dc-4957-82d1-78dac65decf0 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:48.511514 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-edcf5043-93dc-4957-82d1-78dac65decf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:48.522880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:48.581789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:48.636057 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 51/103] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:05:48.640345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfae4f8-3e54-4625-a02b-6e3837487586 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:48.641641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfae4f8-3e54-4625-a02b-6e3837487586 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:48.642113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfae4f8-3e54-4625-a02b-6e3837487586 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:48.644644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 52/104] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:48.651142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e72fe-dece-4c12-a8e3-1a6a5acfe516 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:48.652361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e72fe-dece-4c12-a8e3-1a6a5acfe516 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:48.653459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e72fe-dece-4c12-a8e3-1a6a5acfe516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:48.672480 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fdf1f-bb8a-43c2-94b2-609466964f7e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:48.680094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e84e72fe-dece-4c12-a8e3-1a6a5acfe516 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:48.681595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e84e72fe-dece-4c12-a8e3-1a6a5acfe516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:48.716597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d95225-1dc5-465e-ac36-91e80df3bad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:48.717955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d95225-1dc5-465e-ac36-91e80df3bad9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:48.718377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d95225-1dc5-465e-ac36-91e80df3bad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:48.718822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d95225-1dc5-465e-ac36-91e80df3bad9 None admin] PATH_INFO: `/v3/users/3003cc1d2f22405cbb7a460726ac432a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:48.737833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d95225-1dc5-465e-ac36-91e80df3bad9 None admin] RBAC: Authorizing `identity:get_user(user_id=3003cc1d2f22405cbb7a460726ac432a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:48.739852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d95225-1dc5-465e-ac36-91e80df3bad9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:48.742076 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 53/105] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:48 2022] GET /identity/v3/users/3003cc1d2f22405cbb7a460726ac432a => generated 289 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:48.746599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d95225-1dc5-465e-ac36-91e80df3bad9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:48.773474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 53/106] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:48.775334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44940e-fcab-47b8-b384-6c2b447d9e78 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:48.775810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44940e-fcab-47b8-b384-6c2b447d9e78 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:48.776546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44940e-fcab-47b8-b384-6c2b447d9e78 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:48.779089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 54/107] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:48.781489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44940e-fcab-47b8-b384-6c2b447d9e78 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:48.878323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:48.894353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d95225-1dc5-465e-ac36-91e80df3bad9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:48.912616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44940e-fcab-47b8-b384-6c2b447d9e78 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:48.925575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:48.937921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0522bcb-7b8c-44d8-ae83-7ad319159430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:48.938734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0522bcb-7b8c-44d8-ae83-7ad319159430 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:48.939026 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0522bcb-7b8c-44d8-ae83-7ad319159430 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:48.939307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0522bcb-7b8c-44d8-ae83-7ad319159430 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:48.940515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0522bcb-7b8c-44d8-ae83-7ad319159430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:48.942009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0522bcb-7b8c-44d8-ae83-7ad319159430 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:48.948398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 54/108] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:48.950433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b462-b36f-42bd-bca7-8847a4e0137b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:48.950727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b462-b36f-42bd-bca7-8847a4e0137b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:48.951010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b462-b36f-42bd-bca7-8847a4e0137b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:48.958446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1edad-6dc0-4148-b35b-1c14817dad50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:48.962678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1edad-6dc0-4148-b35b-1c14817dad50 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:48.964153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1edad-6dc0-4148-b35b-1c14817dad50 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:48.964810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1edad-6dc0-4148-b35b-1c14817dad50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:48.970789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8397b462-b36f-42bd-bca7-8847a4e0137b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:48.972235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8397b462-b36f-42bd-bca7-8847a4e0137b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:49.052405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 55/109] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:49.053815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397b462-b36f-42bd-bca7-8847a4e0137b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:49.072030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1edad-6dc0-4148-b35b-1c14817dad50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:49.074619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1edad-6dc0-4148-b35b-1c14817dad50 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:49.163909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:49.173780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397b462-b36f-42bd-bca7-8847a4e0137b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:49.189813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 55/110] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:05:49.192112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1edad-6dc0-4148-b35b-1c14817dad50 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:49.203577 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:49.215218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce815d-d213-49f2-ad49-7ba104b53140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:49.216032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce815d-d213-49f2-ad49-7ba104b53140 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.216315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce815d-d213-49f2-ad49-7ba104b53140 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.216594 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce815d-d213-49f2-ad49-7ba104b53140 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.228076 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce815d-d213-49f2-ad49-7ba104b53140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-30ce815d-d213-49f2-ad49-7ba104b53140 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:49.236471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:49.237345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:05:49.238131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 56/111] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:49.238980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129610b9-6394-4fb3-9f74-49fee208eb91 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.239245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129610b9-6394-4fb3-9f74-49fee208eb91 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.239523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129610b9-6394-4fb3-9f74-49fee208eb91 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.240834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 57/112] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:49.242497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129610b9-6394-4fb3-9f74-49fee208eb91 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:49.301433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1edad-6dc0-4148-b35b-1c14817dad50 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:49.336212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef21924-d962-4596-b156-5226da890f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:49.336879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef21924-d962-4596-b156-5226da890f4a None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.337175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef21924-d962-4596-b156-5226da890f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.337456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef21924-d962-4596-b156-5226da890f4a None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.338190 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef21924-d962-4596-b156-5226da890f4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:49.339636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef21924-d962-4596-b156-5226da890f4a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:49.349213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129610b9-6394-4fb3-9f74-49fee208eb91 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:49.365359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 56/113] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:05:49 2022] POST /identity/v3/projects => generated 302 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:05:49.366888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edae3de-7d2e-4e69-b30f-76634f6e06ad None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.367185 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edae3de-7d2e-4e69-b30f-76634f6e06ad None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.367508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edae3de-7d2e-4e69-b30f-76634f6e06ad None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.368810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 57/114] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:49.369637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edae3de-7d2e-4e69-b30f-76634f6e06ad None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:49.392332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c992038-ac75-4537-8f6a-089caa4a6620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:49.393331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c992038-ac75-4537-8f6a-089caa4a6620 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.393622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c992038-ac75-4537-8f6a-089caa4a6620 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.394053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c992038-ac75-4537-8f6a-089caa4a6620 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.408165 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c992038-ac75-4537-8f6a-089caa4a6620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2c992038-ac75-4537-8f6a-089caa4a6620 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:49.417771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:49.418714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:05:49.419510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 58/115] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:05:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:49.420175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c992038-ac75-4537-8f6a-089caa4a6620 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:49.438454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:49.450945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:49.504401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edae3de-7d2e-4e69-b30f-76634f6e06ad None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:49.543493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c992038-ac75-4537-8f6a-089caa4a6620 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:49.552205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6414ea-4d92-446e-afd8-0839f7ca4540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:49.553647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6414ea-4d92-446e-afd8-0839f7ca4540 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.553974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6414ea-4d92-446e-afd8-0839f7ca4540 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.554260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6414ea-4d92-446e-afd8-0839f7ca4540 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.557329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6414ea-4d92-446e-afd8-0839f7ca4540 None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:49.558826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6414ea-4d92-446e-afd8-0839f7ca4540 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:49.561508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 58/116] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:49 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:49.562563 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587ea54-585e-403d-9d0e-168cfa5e9ade None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.562859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587ea54-585e-403d-9d0e-168cfa5e9ade None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.563146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587ea54-585e-403d-9d0e-168cfa5e9ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.580186 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4587ea54-585e-403d-9d0e-168cfa5e9ade None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:49.581406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4587ea54-585e-403d-9d0e-168cfa5e9ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:49.589213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258370ea-8cbf-457d-bef8-82f05c120d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:49.590588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258370ea-8cbf-457d-bef8-82f05c120d67 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.591037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258370ea-8cbf-457d-bef8-82f05c120d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.591324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258370ea-8cbf-457d-bef8-82f05c120d67 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.592485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258370ea-8cbf-457d-bef8-82f05c120d67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:49.593368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258370ea-8cbf-457d-bef8-82f05c120d67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:49.596094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258370ea-8cbf-457d-bef8-82f05c120d67 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:49.606055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 59/117] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:05:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:49.607354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dc53e-6767-4dec-b44b-eb55027952d7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.607652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dc53e-6767-4dec-b44b-eb55027952d7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.607933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dc53e-6767-4dec-b44b-eb55027952d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.628006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-968dc53e-6767-4dec-b44b-eb55027952d7 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:49.629210 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-968dc53e-6767-4dec-b44b-eb55027952d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:49.657542 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 59/118] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:49.658983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587ea54-585e-403d-9d0e-168cfa5e9ade None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:49.709430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 60/119] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:49.710387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968dc53e-6767-4dec-b44b-eb55027952d7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:49.712767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:49.743042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:49.790895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587ea54-585e-403d-9d0e-168cfa5e9ade None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:49.823396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968dc53e-6767-4dec-b44b-eb55027952d7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:49.831479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946d954-19b0-4323-8e47-5c4fbcce80de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:49.832155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946d954-19b0-4323-8e47-5c4fbcce80de None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.832436 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946d954-19b0-4323-8e47-5c4fbcce80de None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.832712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946d954-19b0-4323-8e47-5c4fbcce80de None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.833458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946d954-19b0-4323-8e47-5c4fbcce80de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:49.833795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946d954-19b0-4323-8e47-5c4fbcce80de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:49.835187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946d954-19b0-4323-8e47-5c4fbcce80de None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:49.844437 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 60/120] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:05:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:49.845060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946d954-19b0-4323-8e47-5c4fbcce80de None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:49.864504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502928c3-ce12-40d1-b093-e256e16656dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:49.866208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502928c3-ce12-40d1-b093-e256e16656dd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.866704 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502928c3-ce12-40d1-b093-e256e16656dd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.867415 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502928c3-ce12-40d1-b093-e256e16656dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.868713 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502928c3-ce12-40d1-b093-e256e16656dd None admin] RBAC: Adding query filter params (role.id=a200b6cb596344e385e5f234488c6018, scope.project.id=82df840b0bc44187a1a9db5c2a668847, user.id=3003cc1d2f22405cbb7a460726ac432a) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:49.869596 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502928c3-ce12-40d1-b093-e256e16656dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:49.871358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502928c3-ce12-40d1-b093-e256e16656dd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:49.885756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 61/121] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:05:49 2022] GET /identity/v3/role_assignments?user.id=3003cc1d2f22405cbb7a460726ac432a&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=a200b6cb596344e385e5f234488c6018 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:49.885756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502928c3-ce12-40d1-b093-e256e16656dd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:49.966162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:49.987623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946d954-19b0-4323-8e47-5c4fbcce80de None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:50.010195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502928c3-ce12-40d1-b093-e256e16656dd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:50.010647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:50.028754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662c6d0-b41e-4a3b-8ac6-528be1d47e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:50.029865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662c6d0-b41e-4a3b-8ac6-528be1d47e96 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:50.030362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662c6d0-b41e-4a3b-8ac6-528be1d47e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:50.030938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662c6d0-b41e-4a3b-8ac6-528be1d47e96 None admin] PATH_INFO: `/v3/users/ab37fe904ac74e65a450dec01100babf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:50.032616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662c6d0-b41e-4a3b-8ac6-528be1d47e96 None admin] RBAC: Authorizing `identity:get_user(user_id=ab37fe904ac74e65a450dec01100babf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:50.034511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662c6d0-b41e-4a3b-8ac6-528be1d47e96 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:50.036405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 61/122] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:49 2022] GET /identity/v3/users/ab37fe904ac74e65a450dec01100babf => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:50.037347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662c6d0-b41e-4a3b-8ac6-528be1d47e96 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:50.052749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65aaaa-47ad-4def-a377-b6f8c7ec2438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:50.053966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65aaaa-47ad-4def-a377-b6f8c7ec2438 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:50.054770 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65aaaa-47ad-4def-a377-b6f8c7ec2438 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:50.055247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65aaaa-47ad-4def-a377-b6f8c7ec2438 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:50.068252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65aaaa-47ad-4def-a377-b6f8c7ec2438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4c65aaaa-47ad-4def-a377-b6f8c7ec2438 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:50.076806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:50.078398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:50.079157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:05:50.081511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 62/123] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:05:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:50.082785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65aaaa-47ad-4def-a377-b6f8c7ec2438 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:50.160944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662c6d0-b41e-4a3b-8ac6-528be1d47e96 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:50.198365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cbcc5-7c52-4fdd-a5b2-dee065c9a6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:50.199339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cbcc5-7c52-4fdd-a5b2-dee065c9a6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:50.199916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cbcc5-7c52-4fdd-a5b2-dee065c9a6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:50.200136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cbcc5-7c52-4fdd-a5b2-dee065c9a6c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:50.215616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168cbcc5-7c52-4fdd-a5b2-dee065c9a6c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:50.220495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65aaaa-47ad-4def-a377-b6f8c7ec2438 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-168cbcc5-7c52-4fdd-a5b2-dee065c9a6c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:50.226483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:50.227494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:50.228460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:50.231714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 62/124] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:50.232847 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cbcc5-7c52-4fdd-a5b2-dee065c9a6c7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:50.260748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:50.272209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538a6f5-f73e-4474-8bd7-1975e6cef2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:50.273264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538a6f5-f73e-4474-8bd7-1975e6cef2ed None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:50.273653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538a6f5-f73e-4474-8bd7-1975e6cef2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:50.274069 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538a6f5-f73e-4474-8bd7-1975e6cef2ed None admin] PATH_INFO: `/v3/users/ab37fe904ac74e65a450dec01100babf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:50.275496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538a6f5-f73e-4474-8bd7-1975e6cef2ed None admin] RBAC: Authorizing `identity:get_user(user_id=ab37fe904ac74e65a450dec01100babf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:50.277358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538a6f5-f73e-4474-8bd7-1975e6cef2ed None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:50.284403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 63/125] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:50 2022] GET /identity/v3/users/ab37fe904ac74e65a450dec01100babf => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:50.284403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538a6f5-f73e-4474-8bd7-1975e6cef2ed None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:50.316020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:50.378984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cbcc5-7c52-4fdd-a5b2-dee065c9a6c7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:50.406506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538a6f5-f73e-4474-8bd7-1975e6cef2ed None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:50.423373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0256d93-86f3-495c-802a-bd4c7971e6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:50.424867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0256d93-86f3-495c-802a-bd4c7971e6de None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:50.425283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0256d93-86f3-495c-802a-bd4c7971e6de None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:50.428022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0256d93-86f3-495c-802a-bd4c7971e6de None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:50.428022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0256d93-86f3-495c-802a-bd4c7971e6de None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:50.428553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0256d93-86f3-495c-802a-bd4c7971e6de None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:50.430539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 63/126] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:50 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:50.436036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0256d93-86f3-495c-802a-bd4c7971e6de None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:50.462429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3666a5e-1da1-4cd7-96a2-a22546125529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:50.463498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3666a5e-1da1-4cd7-96a2-a22546125529 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:50.464569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3666a5e-1da1-4cd7-96a2-a22546125529 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:50.465011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3666a5e-1da1-4cd7-96a2-a22546125529 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:50.466030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3666a5e-1da1-4cd7-96a2-a22546125529 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:50.466482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3666a5e-1da1-4cd7-96a2-a22546125529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:50.469336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3666a5e-1da1-4cd7-96a2-a22546125529 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:50.481056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 64/127] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:50.482446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3666a5e-1da1-4cd7-96a2-a22546125529 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:50.568331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:50.573999 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0256d93-86f3-495c-802a-bd4c7971e6de None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:50.602238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:50.628133 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a625c-c5bc-41b0-877c-28e3a12d2e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:50.628826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a625c-c5bc-41b0-877c-28e3a12d2e8a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:50.629118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a625c-c5bc-41b0-877c-28e3a12d2e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:50.629403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a625c-c5bc-41b0-877c-28e3a12d2e8a None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:50.630185 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a625c-c5bc-41b0-877c-28e3a12d2e8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:50.630499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a625c-c5bc-41b0-877c-28e3a12d2e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:50.631939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a625c-c5bc-41b0-877c-28e3a12d2e8a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:50.638448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3666a5e-1da1-4cd7-96a2-a22546125529 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:50.645829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 64/128] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:05:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:50.646780 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a625c-c5bc-41b0-877c-28e3a12d2e8a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:50.682514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab99fc-268a-4abe-9c94-3900c682499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:50.683774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab99fc-268a-4abe-9c94-3900c682499f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:50.684281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab99fc-268a-4abe-9c94-3900c682499f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:50.684892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab99fc-268a-4abe-9c94-3900c682499f None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:50.687834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab99fc-268a-4abe-9c94-3900c682499f None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:50.689691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab99fc-268a-4abe-9c94-3900c682499f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:50.692212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 65/129] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:50 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:50.693336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab99fc-268a-4abe-9c94-3900c682499f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:50.766926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a625c-c5bc-41b0-877c-28e3a12d2e8a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:50.811086 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9ad18-0df5-40db-94ea-210652877207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:50.813068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9ad18-0df5-40db-94ea-210652877207 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:50.813584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9ad18-0df5-40db-94ea-210652877207 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:50.814307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9ad18-0df5-40db-94ea-210652877207 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847/users/ab37fe904ac74e65a450dec01100babf/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:50.817440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9ad18-0df5-40db-94ea-210652877207 None admin] RBAC: Authorizing `identity:create_grant(project_id=82df840b0bc44187a1a9db5c2a668847, user_id=ab37fe904ac74e65a450dec01100babf, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:50.819095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9ad18-0df5-40db-94ea-210652877207 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:50.819450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab99fc-268a-4abe-9c94-3900c682499f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:50.837391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 65/130] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:05:50 2022] PUT /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847/users/ab37fe904ac74e65a450dec01100babf/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:05:50.838635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9ad18-0df5-40db-94ea-210652877207 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:50.867402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:50.884485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a291b03-2e93-421d-8bc7-e34b4c3c63e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:50.885782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a291b03-2e93-421d-8bc7-e34b4c3c63e6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:50.887073 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a291b03-2e93-421d-8bc7-e34b4c3c63e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:50.887621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a291b03-2e93-421d-8bc7-e34b4c3c63e6 None admin] PATH_INFO: `/v3/users/cada5de94a8d4bee93afa64a6b5b9d33` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:50.889362 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a291b03-2e93-421d-8bc7-e34b4c3c63e6 None admin] RBAC: Authorizing `identity:get_user(user_id=cada5de94a8d4bee93afa64a6b5b9d33)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:50.891311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a291b03-2e93-421d-8bc7-e34b4c3c63e6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:50.893279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 66/131] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:50 2022] GET /identity/v3/users/cada5de94a8d4bee93afa64a6b5b9d33 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:50.894409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a291b03-2e93-421d-8bc7-e34b4c3c63e6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:50.962170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:51.021975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a291b03-2e93-421d-8bc7-e34b4c3c63e6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:51.072385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832a65c-193a-4d6b-88cf-6513b20a4db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:51.073691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832a65c-193a-4d6b-88cf-6513b20a4db9 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:51.074091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832a65c-193a-4d6b-88cf-6513b20a4db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:51.074375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832a65c-193a-4d6b-88cf-6513b20a4db9 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847/users/ab37fe904ac74e65a450dec01100babf/roles/e7180b44fc2945568dfb83dfaf8b5c79` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:51.078311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832a65c-193a-4d6b-88cf-6513b20a4db9 None admin] RBAC: Authorizing `identity:create_grant(project_id=82df840b0bc44187a1a9db5c2a668847, user_id=ab37fe904ac74e65a450dec01100babf, role_id=e7180b44fc2945568dfb83dfaf8b5c79)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:51.080091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832a65c-193a-4d6b-88cf-6513b20a4db9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:51.090710 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9ad18-0df5-40db-94ea-210652877207 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:51.103771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 67/132] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:05:50 2022] PUT /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847/users/ab37fe904ac74e65a450dec01100babf/roles/e7180b44fc2945568dfb83dfaf8b5c79 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:05:51.105051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832a65c-193a-4d6b-88cf-6513b20a4db9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:51.170228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8466d-12dd-4e17-a794-fe1b2d1597bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:51.174166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8466d-12dd-4e17-a794-fe1b2d1597bf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:51.175861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8466d-12dd-4e17-a794-fe1b2d1597bf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:51.176397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8466d-12dd-4e17-a794-fe1b2d1597bf None admin] PATH_INFO: `/v3/users/ab37fe904ac74e65a450dec01100babf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:51.180252 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8466d-12dd-4e17-a794-fe1b2d1597bf None admin] RBAC: Authorizing `identity:get_user(user_id=ab37fe904ac74e65a450dec01100babf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:51.183193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8466d-12dd-4e17-a794-fe1b2d1597bf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:51.188719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 66/133] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:50 2022] GET /identity/v3/users/ab37fe904ac74e65a450dec01100babf => generated 275 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:51.192655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8466d-12dd-4e17-a794-fe1b2d1597bf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:51.222158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:51.332598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832a65c-193a-4d6b-88cf-6513b20a4db9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:51.411327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b39725-d9f5-40a2-ab1c-15b4c632980d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:51.412217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b39725-d9f5-40a2-ab1c-15b4c632980d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:51.412505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b39725-d9f5-40a2-ab1c-15b4c632980d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:51.412789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b39725-d9f5-40a2-ab1c-15b4c632980d None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:51.416554 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8466d-12dd-4e17-a794-fe1b2d1597bf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:51.418636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b39725-d9f5-40a2-ab1c-15b4c632980d None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:51.426865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b39725-d9f5-40a2-ab1c-15b4c632980d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:51.427570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:51.430383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 68/134] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:51 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:51.437447 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b39725-d9f5-40a2-ab1c-15b4c632980d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:51.467556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60508726-74ed-42a9-a098-96d5e3fb0615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:51.468818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60508726-74ed-42a9-a098-96d5e3fb0615 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:51.469226 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60508726-74ed-42a9-a098-96d5e3fb0615 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:51.469510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60508726-74ed-42a9-a098-96d5e3fb0615 None admin] PATH_INFO: `/v3/projects/dcb2341a27ec40efbf28022f6b98ce9a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:51.471722 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60508726-74ed-42a9-a098-96d5e3fb0615 None admin] RBAC: Authorizing `identity:get_project(project_id=dcb2341a27ec40efbf28022f6b98ce9a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:51.473221 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60508726-74ed-42a9-a098-96d5e3fb0615 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:51.476390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 67/135] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:51 2022] GET /identity/v3/projects/dcb2341a27ec40efbf28022f6b98ce9a => generated 312 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:51.481875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60508726-74ed-42a9-a098-96d5e3fb0615 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:51.672371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b39725-d9f5-40a2-ab1c-15b4c632980d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:51.674316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60508726-74ed-42a9-a098-96d5e3fb0615 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:51.730907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:51.774708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa88f59-cc15-4689-a842-8e08b33426e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:51.775625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa88f59-cc15-4689-a842-8e08b33426e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:51.775974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa88f59-cc15-4689-a842-8e08b33426e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:51.776260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa88f59-cc15-4689-a842-8e08b33426e6 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847/users/cada5de94a8d4bee93afa64a6b5b9d33/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:51.778511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa88f59-cc15-4689-a842-8e08b33426e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=82df840b0bc44187a1a9db5c2a668847, user_id=cada5de94a8d4bee93afa64a6b5b9d33, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:51.779090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6295116-1703-4862-8162-d51c0e654a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:51.779983 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa88f59-cc15-4689-a842-8e08b33426e6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:51.780625 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6295116-1703-4862-8162-d51c0e654a17 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:51.781275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6295116-1703-4862-8162-d51c0e654a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:51.781559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6295116-1703-4862-8162-d51c0e654a17 None admin] PATH_INFO: `/v3/projects/dcb2341a27ec40efbf28022f6b98ce9a/users/ab37fe904ac74e65a450dec01100babf/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:51.794624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6295116-1703-4862-8162-d51c0e654a17 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcb2341a27ec40efbf28022f6b98ce9a, user_id=ab37fe904ac74e65a450dec01100babf, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:51.796293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6295116-1703-4862-8162-d51c0e654a17 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:51.806499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 69/136] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:05:51 2022] PUT /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847/users/cada5de94a8d4bee93afa64a6b5b9d33/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:05:51.825425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 68/137] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:05:51 2022] PUT /identity/v3/projects/dcb2341a27ec40efbf28022f6b98ce9a/users/ab37fe904ac74e65a450dec01100babf/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:05:52.075324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a63ac-3e4e-447d-8707-f2527dbed883 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:52.075851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a63ac-3e4e-447d-8707-f2527dbed883 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:52.075925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a63ac-3e4e-447d-8707-f2527dbed883 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:52.077687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 70/138] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:52.093678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe338d92-5bc4-49dc-b97b-c88dd4d0fdb8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:52.094648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe338d92-5bc4-49dc-b97b-c88dd4d0fdb8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:52.095165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe338d92-5bc4-49dc-b97b-c88dd4d0fdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:52.118977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fe338d92-5bc4-49dc-b97b-c88dd4d0fdb8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:52.120565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fe338d92-5bc4-49dc-b97b-c88dd4d0fdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:52.158447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:52.207937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 69/139] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:52.234902 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a63ac-3e4e-447d-8707-f2527dbed883 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:52.388287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a63ac-3e4e-447d-8707-f2527dbed883 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:52.445407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c684d-00c8-42ef-9106-45903190f9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:52.446280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c684d-00c8-42ef-9106-45903190f9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:52.446567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c684d-00c8-42ef-9106-45903190f9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:52.446845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c684d-00c8-42ef-9106-45903190f9a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:52.448067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c684d-00c8-42ef-9106-45903190f9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:52.449549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c684d-00c8-42ef-9106-45903190f9a3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:52.451453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 71/140] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:52.457774 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe338d92-5bc4-49dc-b97b-c88dd4d0fdb8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:52.599829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe338d92-5bc4-49dc-b97b-c88dd4d0fdb8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:52.645293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679d319-55c7-4ac8-850f-2d95d913dd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:52.646415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679d319-55c7-4ac8-850f-2d95d913dd12 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:52.646907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679d319-55c7-4ac8-850f-2d95d913dd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:52.647386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679d319-55c7-4ac8-850f-2d95d913dd12 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:52.648318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679d319-55c7-4ac8-850f-2d95d913dd12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:52.650008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679d319-55c7-4ac8-850f-2d95d913dd12 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:52.684251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 70/141] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:05:52 2022] POST /identity/v3/users => generated 283 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:05:52.784303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904658e3-a3f1-4146-b0f7-ec9071638b0a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:52.784621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904658e3-a3f1-4146-b0f7-ec9071638b0a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:52.784876 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904658e3-a3f1-4146-b0f7-ec9071638b0a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:52.786593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 72/142] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:52.792453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fbb85-aba6-460b-91a3-8b3c620b4a43 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:52.793041 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fbb85-aba6-460b-91a3-8b3c620b4a43 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:52.793521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fbb85-aba6-460b-91a3-8b3c620b4a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:52.814994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d60fbb85-aba6-460b-91a3-8b3c620b4a43 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:52.816530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d60fbb85-aba6-460b-91a3-8b3c620b4a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:52.833450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:52.908935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 71/143] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:52.938099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904658e3-a3f1-4146-b0f7-ec9071638b0a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:53.015329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:53.083342 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904658e3-a3f1-4146-b0f7-ec9071638b0a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:53.128226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143e9b3-ddb7-4cba-bfb4-b4678ac385d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:53.129043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143e9b3-ddb7-4cba-bfb4-b4678ac385d0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:53.129332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143e9b3-ddb7-4cba-bfb4-b4678ac385d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:53.129893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143e9b3-ddb7-4cba-bfb4-b4678ac385d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:53.150382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143e9b3-ddb7-4cba-bfb4-b4678ac385d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0143e9b3-ddb7-4cba-bfb4-b4678ac385d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:53.158386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:53.159538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:05:53.160901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 73/144] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:53.165654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fbb85-aba6-460b-91a3-8b3c620b4a43 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:53.303159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fbb85-aba6-460b-91a3-8b3c620b4a43 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:53.349541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf0de1-dcdb-4182-ba90-59b0436faf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:53.350288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0de1-dcdb-4182-ba90-59b0436faf26 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:53.350559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0de1-dcdb-4182-ba90-59b0436faf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:53.350837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0de1-dcdb-4182-ba90-59b0436faf26 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:53.351606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf0de1-dcdb-4182-ba90-59b0436faf26 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:53.351919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf0de1-dcdb-4182-ba90-59b0436faf26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:53.353505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf0de1-dcdb-4182-ba90-59b0436faf26 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:53.362787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 72/145] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:05:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:53.368687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143e9b3-ddb7-4cba-bfb4-b4678ac385d0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:53.490307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143e9b3-ddb7-4cba-bfb4-b4678ac385d0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:53.502590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:53.538089 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cf29e-bc66-47b9-9e7b-7eea106d2e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:53.538992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cf29e-bc66-47b9-9e7b-7eea106d2e00 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:53.539275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cf29e-bc66-47b9-9e7b-7eea106d2e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:53.539562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cf29e-bc66-47b9-9e7b-7eea106d2e00 None admin] PATH_INFO: `/v3/users/3003cc1d2f22405cbb7a460726ac432a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:53.542519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cf29e-bc66-47b9-9e7b-7eea106d2e00 None admin] RBAC: Authorizing `identity:get_user(user_id=3003cc1d2f22405cbb7a460726ac432a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:53.544228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cf29e-bc66-47b9-9e7b-7eea106d2e00 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:53.546468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 74/146] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:53 2022] GET /identity/v3/users/3003cc1d2f22405cbb7a460726ac432a => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:53.552452 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf0de1-dcdb-4182-ba90-59b0436faf26 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:53.575741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:53.697359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf0de1-dcdb-4182-ba90-59b0436faf26 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:53.741091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e554705-0d1d-4db8-89ef-dec045b473c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:53.742040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e554705-0d1d-4db8-89ef-dec045b473c8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:53.742330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e554705-0d1d-4db8-89ef-dec045b473c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:53.742615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e554705-0d1d-4db8-89ef-dec045b473c8 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:53.743899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e554705-0d1d-4db8-89ef-dec045b473c8 None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:53.745405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e554705-0d1d-4db8-89ef-dec045b473c8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:53.747494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 73/147] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:53 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:53.758843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cf29e-bc66-47b9-9e7b-7eea106d2e00 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:53.923164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cf29e-bc66-47b9-9e7b-7eea106d2e00 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:53.973774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7964d2-fc75-4e95-b259-ad4d8212bc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:53.974631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7964d2-fc75-4e95-b259-ad4d8212bc28 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:53.974915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7964d2-fc75-4e95-b259-ad4d8212bc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:53.975190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7964d2-fc75-4e95-b259-ad4d8212bc28 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847/users/3003cc1d2f22405cbb7a460726ac432a/roles/a200b6cb596344e385e5f234488c6018` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:53.984110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7964d2-fc75-4e95-b259-ad4d8212bc28 None admin] RBAC: Authorizing `identity:create_grant(project_id=82df840b0bc44187a1a9db5c2a668847, user_id=3003cc1d2f22405cbb7a460726ac432a, role_id=a200b6cb596344e385e5f234488c6018)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:53.985780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7964d2-fc75-4e95-b259-ad4d8212bc28 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:54.007798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 75/148] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:05:53 2022] PUT /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847/users/3003cc1d2f22405cbb7a460726ac432a/roles/a200b6cb596344e385e5f234488c6018 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:05:54.287447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650f217-fa35-49ac-a2cb-cb29c040413f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:54.287933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650f217-fa35-49ac-a2cb-cb29c040413f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:54.287933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650f217-fa35-49ac-a2cb-cb29c040413f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:54.289365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 74/149] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:54.295700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b8ee6-3668-4cdb-8d0d-6c0874b7aed4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:54.296323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b8ee6-3668-4cdb-8d0d-6c0874b7aed4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:54.296323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b8ee6-3668-4cdb-8d0d-6c0874b7aed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:54.319466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-db4b8ee6-3668-4cdb-8d0d-6c0874b7aed4 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:54.320679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-db4b8ee6-3668-4cdb-8d0d-6c0874b7aed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:54.358303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:54.405880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 76/150] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:54.442726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650f217-fa35-49ac-a2cb-cb29c040413f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:54.518435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1883a8-3f15-4f11-8a2c-0c0b2e8a451b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:54.518772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1883a8-3f15-4f11-8a2c-0c0b2e8a451b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:54.519175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1883a8-3f15-4f11-8a2c-0c0b2e8a451b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:54.521216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 77/151] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:05:54.530140 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95106f1-5712-47c9-838b-a2fd340cc979 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:54.530590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95106f1-5712-47c9-838b-a2fd340cc979 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:54.530833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95106f1-5712-47c9-838b-a2fd340cc979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:54.552469 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d95106f1-5712-47c9-838b-a2fd340cc979 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:54.554833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d95106f1-5712-47c9-838b-a2fd340cc979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:54.562332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:54.582977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650f217-fa35-49ac-a2cb-cb29c040413f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:54.626100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f277b-a8e6-449f-ae62-14cfa16a99b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:54.627060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f277b-a8e6-449f-ae62-14cfa16a99b4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:54.627346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f277b-a8e6-449f-ae62-14cfa16a99b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:54.627619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f277b-a8e6-449f-ae62-14cfa16a99b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:54.648514 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f277b-a8e6-449f-ae62-14cfa16a99b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-304f277b-a8e6-449f-ae62-14cfa16a99b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:54.657011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:54.657934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:54.658942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:54.662490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 75/152] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:54.668304 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f277b-a8e6-449f-ae62-14cfa16a99b4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:54.703980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 78/153] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:54.765354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95106f1-5712-47c9-838b-a2fd340cc979 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:54.879821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f277b-a8e6-449f-ae62-14cfa16a99b4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:54.982202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:54.986553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf7a27-4bba-4958-a03d-91e43162a260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:54.987260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf7a27-4bba-4958-a03d-91e43162a260 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:54.987542 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf7a27-4bba-4958-a03d-91e43162a260 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:54.987817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf7a27-4bba-4958-a03d-91e43162a260 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:54.988580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf7a27-4bba-4958-a03d-91e43162a260 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:54.988884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf7a27-4bba-4958-a03d-91e43162a260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:54.989890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95106f1-5712-47c9-838b-a2fd340cc979 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:54.990411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf7a27-4bba-4958-a03d-91e43162a260 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:55.005692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 76/154] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:05:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:55.011138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf7a27-4bba-4958-a03d-91e43162a260 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:55.016997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:55.082497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bff23f-ff7a-45fd-8a8e-d424c21f9010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:55.083346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bff23f-ff7a-45fd-8a8e-d424c21f9010 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:55.083634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bff23f-ff7a-45fd-8a8e-d424c21f9010 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:55.083913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bff23f-ff7a-45fd-8a8e-d424c21f9010 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:55.111064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bff23f-ff7a-45fd-8a8e-d424c21f9010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-41bff23f-ff7a-45fd-8a8e-d424c21f9010 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:55.123873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:55.124802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:05:55.125741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 79/155] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:05:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:55.133158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bff23f-ff7a-45fd-8a8e-d424c21f9010 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:55.306433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf7a27-4bba-4958-a03d-91e43162a260 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:55.357752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bff23f-ff7a-45fd-8a8e-d424c21f9010 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:55.402591 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22daaf-f8c6-43fd-a7cb-84179ec6c677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:55.403397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22daaf-f8c6-43fd-a7cb-84179ec6c677 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:55.403684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22daaf-f8c6-43fd-a7cb-84179ec6c677 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:55.403963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22daaf-f8c6-43fd-a7cb-84179ec6c677 None admin] PATH_INFO: `/v3/users/ab37fe904ac74e65a450dec01100babf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:55.407869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22daaf-f8c6-43fd-a7cb-84179ec6c677 None admin] RBAC: Authorizing `identity:get_user(user_id=ab37fe904ac74e65a450dec01100babf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:55.409604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22daaf-f8c6-43fd-a7cb-84179ec6c677 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:55.419604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 77/156] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:55 2022] GET /identity/v3/users/ab37fe904ac74e65a450dec01100babf => generated 275 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:55.426013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22daaf-f8c6-43fd-a7cb-84179ec6c677 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:55.454616 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43b00c-f83f-43b6-abaa-688b5e191201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:55.455769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43b00c-f83f-43b6-abaa-688b5e191201 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:55.456059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43b00c-f83f-43b6-abaa-688b5e191201 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:55.456341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43b00c-f83f-43b6-abaa-688b5e191201 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:55.457119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43b00c-f83f-43b6-abaa-688b5e191201 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:55.457432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43b00c-f83f-43b6-abaa-688b5e191201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:55.463168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43b00c-f83f-43b6-abaa-688b5e191201 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:55.473144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 80/157] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:05:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:55.478957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43b00c-f83f-43b6-abaa-688b5e191201 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:55.506104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:55.554132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:55.607965 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22daaf-f8c6-43fd-a7cb-84179ec6c677 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:55.651747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43b00c-f83f-43b6-abaa-688b5e191201 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:55.682548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de0872-3350-4679-a7bb-a769b7152d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:55.683150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de0872-3350-4679-a7bb-a769b7152d67 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:55.683439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de0872-3350-4679-a7bb-a769b7152d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:55.683721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de0872-3350-4679-a7bb-a769b7152d67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:55.700899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59810523-d26b-461d-af85-e2273091ea49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:55.702444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59810523-d26b-461d-af85-e2273091ea49 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:55.702886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59810523-d26b-461d-af85-e2273091ea49 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:55.703292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59810523-d26b-461d-af85-e2273091ea49 None admin] PATH_INFO: `/v3/users/ab37fe904ac74e65a450dec01100babf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:55.704578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59810523-d26b-461d-af85-e2273091ea49 None admin] RBAC: Authorizing `identity:get_user(user_id=ab37fe904ac74e65a450dec01100babf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:55.710419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59810523-d26b-461d-af85-e2273091ea49 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:55.712176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 81/158] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:55 2022] GET /identity/v3/users/ab37fe904ac74e65a450dec01100babf => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:55.717795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59810523-d26b-461d-af85-e2273091ea49 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:55.802870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de0872-3350-4679-a7bb-a769b7152d67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:55.804372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de0872-3350-4679-a7bb-a769b7152d67 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:55.862137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:55.875588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59810523-d26b-461d-af85-e2273091ea49 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:55.933552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09231707-7f7d-4d33-95ff-d372123b1be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:55.934156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09231707-7f7d-4d33-95ff-d372123b1be6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:55.934440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09231707-7f7d-4d33-95ff-d372123b1be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:55.934724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09231707-7f7d-4d33-95ff-d372123b1be6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.031197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09231707-7f7d-4d33-95ff-d372123b1be6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:56.032709 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09231707-7f7d-4d33-95ff-d372123b1be6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:56.049395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:56.127321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 78/159] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:05:56.128766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043d497-b73e-4466-80a9-3d8300f29fbe None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.129057 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043d497-b73e-4466-80a9-3d8300f29fbe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.138201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043d497-b73e-4466-80a9-3d8300f29fbe None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.140615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 79/160] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:56 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:56.141557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043d497-b73e-4466-80a9-3d8300f29fbe None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:56.161547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:56.181628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 82/161] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 16:05:56.182886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae4b55-a6ce-4d12-b153-525fcded612d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.183185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae4b55-a6ce-4d12-b153-525fcded612d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.186238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae4b55-a6ce-4d12-b153-525fcded612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.211327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-06ae4b55-a6ce-4d12-b153-525fcded612d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:56.212553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-06ae4b55-a6ce-4d12-b153-525fcded612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:56.315803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 83/162] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:56.316763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae4b55-a6ce-4d12-b153-525fcded612d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:56.334561 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043d497-b73e-4466-80a9-3d8300f29fbe None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:56.406784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0d894-8a74-4a67-84ae-a8bada1b4bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:56.407683 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0d894-8a74-4a67-84ae-a8bada1b4bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.407966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0d894-8a74-4a67-84ae-a8bada1b4bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.408248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0d894-8a74-4a67-84ae-a8bada1b4bc9 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.409792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0d894-8a74-4a67-84ae-a8bada1b4bc9 None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:56.411250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0d894-8a74-4a67-84ae-a8bada1b4bc9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:56.412971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 80/163] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:56 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:56.413756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0d894-8a74-4a67-84ae-a8bada1b4bc9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:56.457688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae4b55-a6ce-4d12-b153-525fcded612d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:56.476181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:56.496319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc1042-cc00-4c56-bde7-6d3736436ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:56.497434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc1042-cc00-4c56-bde7-6d3736436ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.497882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc1042-cc00-4c56-bde7-6d3736436ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.498328 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc1042-cc00-4c56-bde7-6d3736436ff2 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.500253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc1042-cc00-4c56-bde7-6d3736436ff2 None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:56.502018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc1042-cc00-4c56-bde7-6d3736436ff2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:56.503928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 84/164] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:56 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:56.509484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc1042-cc00-4c56-bde7-6d3736436ff2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:56.530264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:56.620531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc1042-cc00-4c56-bde7-6d3736436ff2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:56.623914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0d894-8a74-4a67-84ae-a8bada1b4bc9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:56.658025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf521-3ba2-47d8-b92d-9dedd8f9c157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:56.659180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf521-3ba2-47d8-b92d-9dedd8f9c157 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.659634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf521-3ba2-47d8-b92d-9dedd8f9c157 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.660050 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf521-3ba2-47d8-b92d-9dedd8f9c157 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.661029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacf521-3ba2-47d8-b92d-9dedd8f9c157 None admin] RBAC: Adding query filter params (role.id=7f6dd280cf8941dcb8f79ecf88dbda78, scope.project.id=82df840b0bc44187a1a9db5c2a668847, user.id=ab37fe904ac74e65a450dec01100babf) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:56.661484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacf521-3ba2-47d8-b92d-9dedd8f9c157 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:56.663107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacf521-3ba2-47d8-b92d-9dedd8f9c157 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:56.669523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 85/165] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:05:56 2022] GET /identity/v3/role_assignments?user.id=ab37fe904ac74e65a450dec01100babf&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:56.671820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8679e3f-f0e3-4d46-8bd3-fb31f4b51607 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.672261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8679e3f-f0e3-4d46-8bd3-fb31f4b51607 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.672682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8679e3f-f0e3-4d46-8bd3-fb31f4b51607 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.674125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 86/166] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:56.675349 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8679e3f-f0e3-4d46-8bd3-fb31f4b51607 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:56.681494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0212c-3a17-41e8-b6f9-df109e6f057b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:56.682346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0212c-3a17-41e8-b6f9-df109e6f057b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.682626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0212c-3a17-41e8-b6f9-df109e6f057b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.682899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0212c-3a17-41e8-b6f9-df109e6f057b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.699652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0212c-3a17-41e8-b6f9-df109e6f057b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-81a0212c-3a17-41e8-b6f9-df109e6f057b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:56.716964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:56.718045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:56.719404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 81/167] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:56.720288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62081870-2d0b-43a3-99c4-a9ab1cec04c8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.720443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62081870-2d0b-43a3-99c4-a9ab1cec04c8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.720917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62081870-2d0b-43a3-99c4-a9ab1cec04c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.738232 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:56.751339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-62081870-2d0b-43a3-99c4-a9ab1cec04c8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:56.752810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-62081870-2d0b-43a3-99c4-a9ab1cec04c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:56.798239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8679e3f-f0e3-4d46-8bd3-fb31f4b51607 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:56.836662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9447a-235f-4af4-929a-d4a40c2c923d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:56.837944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9447a-235f-4af4-929a-d4a40c2c923d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.838389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9447a-235f-4af4-929a-d4a40c2c923d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.838808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9447a-235f-4af4-929a-d4a40c2c923d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.839799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9447a-235f-4af4-929a-d4a40c2c923d None admin] RBAC: Adding query filter params (role.id=e7180b44fc2945568dfb83dfaf8b5c79, scope.project.id=82df840b0bc44187a1a9db5c2a668847, user.id=ab37fe904ac74e65a450dec01100babf) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:56.840277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9447a-235f-4af4-929a-d4a40c2c923d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:56.841897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9447a-235f-4af4-929a-d4a40c2c923d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:56.848946 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 87/168] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:05:56 2022] GET /identity/v3/role_assignments?user.id=ab37fe904ac74e65a450dec01100babf&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=e7180b44fc2945568dfb83dfaf8b5c79 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:56.854851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9447a-235f-4af4-929a-d4a40c2c923d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:56.924129 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 82/169] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:56.969349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62081870-2d0b-43a3-99c4-a9ab1cec04c8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:56.995280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9447a-235f-4af4-929a-d4a40c2c923d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:57.010316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:57.034418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb259b-648d-4f1d-ae1c-448f9d6dcc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:57.035378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb259b-648d-4f1d-ae1c-448f9d6dcc40 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:57.035817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb259b-648d-4f1d-ae1c-448f9d6dcc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:57.036248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb259b-648d-4f1d-ae1c-448f9d6dcc40 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:57.037217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb259b-648d-4f1d-ae1c-448f9d6dcc40 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:57.037841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb259b-648d-4f1d-ae1c-448f9d6dcc40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:57.039701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb259b-648d-4f1d-ae1c-448f9d6dcc40 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:57.048974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 88/170] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:05:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:57.055561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb259b-648d-4f1d-ae1c-448f9d6dcc40 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:57.058108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:57.190992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62081870-2d0b-43a3-99c4-a9ab1cec04c8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:57.218800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb259b-648d-4f1d-ae1c-448f9d6dcc40 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:57.265038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703cf45-518d-4e09-9b51-54568ba6c679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:57.265888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703cf45-518d-4e09-9b51-54568ba6c679 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:57.266390 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703cf45-518d-4e09-9b51-54568ba6c679 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:57.266664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703cf45-518d-4e09-9b51-54568ba6c679 None admin] PATH_INFO: `/v3/users/ab37fe904ac74e65a450dec01100babf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:57.268156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703cf45-518d-4e09-9b51-54568ba6c679 None admin] RBAC: Authorizing `identity:get_user(user_id=ab37fe904ac74e65a450dec01100babf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:57.269872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703cf45-518d-4e09-9b51-54568ba6c679 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:57.271479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af94b4-3907-4740-ab5f-3f3fac383508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:57.272156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 89/171] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:57 2022] GET /identity/v3/users/ab37fe904ac74e65a450dec01100babf => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:57.272366 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af94b4-3907-4740-ab5f-3f3fac383508 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:57.272618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af94b4-3907-4740-ab5f-3f3fac383508 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:57.272896 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af94b4-3907-4740-ab5f-3f3fac383508 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:57.279138 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703cf45-518d-4e09-9b51-54568ba6c679 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:57.306499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af94b4-3907-4740-ab5f-3f3fac383508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b0af94b4-3907-4740-ab5f-3f3fac383508 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:57.315712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:57.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:57.317427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:57.319907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 83/172] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:05:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:05:57.325897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af94b4-3907-4740-ab5f-3f3fac383508 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:57.336574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:57.404213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703cf45-518d-4e09-9b51-54568ba6c679 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:57.453135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13405227-0311-43ac-98f2-c71d7f2f9f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:57.453935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13405227-0311-43ac-98f2-c71d7f2f9f7c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:57.454386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13405227-0311-43ac-98f2-c71d7f2f9f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:57.454820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13405227-0311-43ac-98f2-c71d7f2f9f7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:57.525220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:57.532011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13405227-0311-43ac-98f2-c71d7f2f9f7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:57.533610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13405227-0311-43ac-98f2-c71d7f2f9f7c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:57.557837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af94b4-3907-4740-ab5f-3f3fac383508 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:57.589988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:57.614478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d701d4-db93-4c74-81a9-b93d1eeca6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:57.615184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d701d4-db93-4c74-81a9-b93d1eeca6df None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:57.615478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d701d4-db93-4c74-81a9-b93d1eeca6df None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:57.615760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d701d4-db93-4c74-81a9-b93d1eeca6df None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:57.616533 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d701d4-db93-4c74-81a9-b93d1eeca6df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:57.616843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d701d4-db93-4c74-81a9-b93d1eeca6df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:57.618290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d701d4-db93-4c74-81a9-b93d1eeca6df None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:57.627884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 84/173] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:57.633510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d701d4-db93-4c74-81a9-b93d1eeca6df None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:57.649059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 90/174] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:05:57.672731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13405227-0311-43ac-98f2-c71d7f2f9f7c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:57.770662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d701d4-db93-4c74-81a9-b93d1eeca6df None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:57.795152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13405227-0311-43ac-98f2-c71d7f2f9f7c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:57.835815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0683d-6796-4792-b441-bc7156c55b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:57.836714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0683d-6796-4792-b441-bc7156c55b2c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:57.837001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0683d-6796-4792-b441-bc7156c55b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:57.837282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0683d-6796-4792-b441-bc7156c55b2c None admin] PATH_INFO: `/v3/users/cada5de94a8d4bee93afa64a6b5b9d33` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:57.846611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0683d-6796-4792-b441-bc7156c55b2c None admin] RBAC: Authorizing `identity:get_user(user_id=cada5de94a8d4bee93afa64a6b5b9d33)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:57.848123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0683d-6796-4792-b441-bc7156c55b2c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:57.850038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 85/175] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:57 2022] GET /identity/v3/users/cada5de94a8d4bee93afa64a6b5b9d33 => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:57.855598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0683d-6796-4792-b441-bc7156c55b2c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:57.861048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8e586-c388-4bb9-8021-4222ee47a94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:57.861948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8e586-c388-4bb9-8021-4222ee47a94c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:57.862236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8e586-c388-4bb9-8021-4222ee47a94c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:57.862512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8e586-c388-4bb9-8021-4222ee47a94c None admin] PATH_INFO: `/v3/projects/dcb2341a27ec40efbf28022f6b98ce9a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:57.863738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8e586-c388-4bb9-8021-4222ee47a94c None admin] RBAC: Authorizing `identity:get_project(project_id=dcb2341a27ec40efbf28022f6b98ce9a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:57.865218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8e586-c388-4bb9-8021-4222ee47a94c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:57.867156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 91/176] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:57 2022] GET /identity/v3/projects/dcb2341a27ec40efbf28022f6b98ce9a => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:57.880952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8e586-c388-4bb9-8021-4222ee47a94c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:57.897801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:57.906083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:58.028062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0683d-6796-4792-b441-bc7156c55b2c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:58.094656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fb49d-27be-4078-acf7-39784ecb913a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:58.095351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fb49d-27be-4078-acf7-39784ecb913a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.095640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fb49d-27be-4078-acf7-39784ecb913a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.095920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fb49d-27be-4078-acf7-39784ecb913a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.157951 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8e586-c388-4bb9-8021-4222ee47a94c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:58.198419 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fb49d-27be-4078-acf7-39784ecb913a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:58.199914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fb49d-27be-4078-acf7-39784ecb913a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:58.235674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:58.240317 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d9aa5-7bc1-4acc-9b12-1be57bdcd2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:58.240995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d9aa5-7bc1-4acc-9b12-1be57bdcd2dd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.241282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d9aa5-7bc1-4acc-9b12-1be57bdcd2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.241565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d9aa5-7bc1-4acc-9b12-1be57bdcd2dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.242449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d9aa5-7bc1-4acc-9b12-1be57bdcd2dd None admin] RBAC: Adding query filter params (role.id=7f6dd280cf8941dcb8f79ecf88dbda78, scope.project.id=dcb2341a27ec40efbf28022f6b98ce9a, user.id=ab37fe904ac74e65a450dec01100babf) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:58.242767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d9aa5-7bc1-4acc-9b12-1be57bdcd2dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:58.244228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d9aa5-7bc1-4acc-9b12-1be57bdcd2dd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:58.251753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 92/177] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:05:57 2022] GET /identity/v3/role_assignments?user.id=ab37fe904ac74e65a450dec01100babf&scope.project.id=dcb2341a27ec40efbf28022f6b98ce9a&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 608 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:05:58.253059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc4b00-bda3-4f59-b621-c55c197d45a4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.253358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc4b00-bda3-4f59-b621-c55c197d45a4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.253643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc4b00-bda3-4f59-b621-c55c197d45a4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.255141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 93/178] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:58.256297 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aebb41-2305-432b-bedf-a7b5c13223df None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.256588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aebb41-2305-432b-bedf-a7b5c13223df None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.256870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aebb41-2305-432b-bedf-a7b5c13223df None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.258639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 94/179] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:58.263338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d8aab-2eb9-4f57-b7d8-a3a51a1d4b8f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.263942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d8aab-2eb9-4f57-b7d8-a3a51a1d4b8f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.264684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d8aab-2eb9-4f57-b7d8-a3a51a1d4b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.284954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-433d8aab-2eb9-4f57-b7d8-a3a51a1d4b8f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:58.287638 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-433d8aab-2eb9-4f57-b7d8-a3a51a1d4b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:58.370805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 86/180] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:05:58.377435 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8191-e40b-4dd0-bdb4-7911deb00891 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.378097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8191-e40b-4dd0-bdb4-7911deb00891 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.378603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8191-e40b-4dd0-bdb4-7911deb00891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.395557 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 95/181] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:58.396941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f4151-e5e8-4593-9af6-2d0221490f16 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.397236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f4151-e5e8-4593-9af6-2d0221490f16 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.397519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f4151-e5e8-4593-9af6-2d0221490f16 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.398582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-020f8191-e40b-4dd0-bdb4-7911deb00891 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:58.398854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 96/182] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:58.399563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f4151-e5e8-4593-9af6-2d0221490f16 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:58.400425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-020f8191-e40b-4dd0-bdb4-7911deb00891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:58.408517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:58.507245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 87/183] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:58.508176 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc97727-f6e5-4cb3-9588-9585e07cbf21 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.508471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc97727-f6e5-4cb3-9588-9585e07cbf21 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.508751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc97727-f6e5-4cb3-9588-9585e07cbf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.532538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3fc97727-f6e5-4cb3-9588-9585e07cbf21 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:58.534012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3fc97727-f6e5-4cb3-9588-9585e07cbf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:58.553775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f4151-e5e8-4593-9af6-2d0221490f16 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:58.590980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:58.601047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774baee-af86-4f64-b4cc-bb8689fe4bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:58.601950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774baee-af86-4f64-b4cc-bb8689fe4bba None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.602230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774baee-af86-4f64-b4cc-bb8689fe4bba None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.602502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774baee-af86-4f64-b4cc-bb8689fe4bba None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.603797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774baee-af86-4f64-b4cc-bb8689fe4bba None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:58.605780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774baee-af86-4f64-b4cc-bb8689fe4bba None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:58.607843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 97/184] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:05:58 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:58.609886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774baee-af86-4f64-b4cc-bb8689fe4bba None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:58.622382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 88/185] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:58.623698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc97727-f6e5-4cb3-9588-9585e07cbf21 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:58.700544 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:58.730515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc97727-f6e5-4cb3-9588-9585e07cbf21 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:58.730942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774baee-af86-4f64-b4cc-bb8689fe4bba None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:58.765373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21e6ef-1cc1-437d-9582-912fc9e32e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:58.765581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1f5da-4558-4368-ae28-50bc1db84d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:58.766205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21e6ef-1cc1-437d-9582-912fc9e32e6b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.766398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1f5da-4558-4368-ae28-50bc1db84d1e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.766488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21e6ef-1cc1-437d-9582-912fc9e32e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.766760 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1f5da-4558-4368-ae28-50bc1db84d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.766878 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21e6ef-1cc1-437d-9582-912fc9e32e6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.766963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1f5da-4558-4368-ae28-50bc1db84d1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.776877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21e6ef-1cc1-437d-9582-912fc9e32e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:58.777004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1f5da-4558-4368-ae28-50bc1db84d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-71e1f5da-4558-4368-ae28-50bc1db84d1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:58.785275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:58.786195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:05:58.786965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1b21e6ef-1cc1-437d-9582-912fc9e32e6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:58.787575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:58.788360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:05:58.789165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:05:58.790995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 98/187] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:05:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:58.791148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 89/187] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:58.791285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1f5da-4558-4368-ae28-50bc1db84d1e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:58.791510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21e6ef-1cc1-437d-9582-912fc9e32e6b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:58.828775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:58.891478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21e6ef-1cc1-437d-9582-912fc9e32e6b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:58.891810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1f5da-4558-4368-ae28-50bc1db84d1e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:58.921350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:58.926447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68dc2a-aba7-4890-9acf-32c14f253151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:58.926651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6922d-0c47-4a7d-8897-971cc121a0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:58.927254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68dc2a-aba7-4890-9acf-32c14f253151 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.927346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6922d-0c47-4a7d-8897-971cc121a0da None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.927540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68dc2a-aba7-4890-9acf-32c14f253151 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.927643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6922d-0c47-4a7d-8897-971cc121a0da None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.927816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68dc2a-aba7-4890-9acf-32c14f253151 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.927903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6922d-0c47-4a7d-8897-971cc121a0da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.928677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6922d-0c47-4a7d-8897-971cc121a0da None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=82df840b0bc44187a1a9db5c2a668847, user.id=cada5de94a8d4bee93afa64a6b5b9d33) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:58.928975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68dc2a-aba7-4890-9acf-32c14f253151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:58.929073 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6922d-0c47-4a7d-8897-971cc121a0da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:58.930483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6922d-0c47-4a7d-8897-971cc121a0da None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:58.930610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68dc2a-aba7-4890-9acf-32c14f253151 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:58.932258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 90/188] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:05:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:58.933417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4273b2d-917d-4266-bd1f-1b6c8a32a1e7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:58.933706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4273b2d-917d-4266-bd1f-1b6c8a32a1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:58.934009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4273b2d-917d-4266-bd1f-1b6c8a32a1e7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:58.935248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 91/189] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:58.937863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4273b2d-917d-4266-bd1f-1b6c8a32a1e7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:58.953793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 99/190] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:05:58 2022] GET /identity/v3/role_assignments?user.id=cada5de94a8d4bee93afa64a6b5b9d33&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:58.953793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6922d-0c47-4a7d-8897-971cc121a0da None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:59.083307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4273b2d-917d-4266-bd1f-1b6c8a32a1e7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:59.087901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6922d-0c47-4a7d-8897-971cc121a0da None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:59.091809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:59.125972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194220a-f805-4352-b543-be6d2ea55922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:59.126667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194220a-f805-4352-b543-be6d2ea55922 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:59.126944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194220a-f805-4352-b543-be6d2ea55922 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:59.127220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194220a-f805-4352-b543-be6d2ea55922 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:59.128014 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194220a-f805-4352-b543-be6d2ea55922 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:59.128284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194220a-f805-4352-b543-be6d2ea55922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:59.129175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9de00-6497-4eab-84fc-1ef198cd07b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:59.129674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194220a-f805-4352-b543-be6d2ea55922 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:59.129926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9de00-6497-4eab-84fc-1ef198cd07b5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:59.130213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9de00-6497-4eab-84fc-1ef198cd07b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:59.130782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9de00-6497-4eab-84fc-1ef198cd07b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:59.131806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9de00-6497-4eab-84fc-1ef198cd07b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:05:59.132128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9de00-6497-4eab-84fc-1ef198cd07b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:59.133840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9de00-6497-4eab-84fc-1ef198cd07b5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:59.140100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 92/191] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:05:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:59.141141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194220a-f805-4352-b543-be6d2ea55922 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:59.145816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 100/192] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:05:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:59.145816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4dd7b-06c4-447a-9c28-e84e5f318329 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:59.145816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4dd7b-06c4-447a-9c28-e84e5f318329 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:59.146058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4dd7b-06c4-447a-9c28-e84e5f318329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:59.164566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-26d4dd7b-06c4-447a-9c28-e84e5f318329 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:59.165765 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-26d4dd7b-06c4-447a-9c28-e84e5f318329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:59.227371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:59.243265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 101/193] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:05:59.244739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4dd7b-06c4-447a-9c28-e84e5f318329 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:59.268700 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194220a-f805-4352-b543-be6d2ea55922 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:59.304375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb3e7f-d483-49b6-97be-fba5626b64f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:59.305062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb3e7f-d483-49b6-97be-fba5626b64f1 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:59.305346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb3e7f-d483-49b6-97be-fba5626b64f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:59.305623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb3e7f-d483-49b6-97be-fba5626b64f1 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:59.306359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb3e7f-d483-49b6-97be-fba5626b64f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:59.307803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb3e7f-d483-49b6-97be-fba5626b64f1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:59.335914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 93/194] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:05:59 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:05:59.335914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb3e7f-d483-49b6-97be-fba5626b64f1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:59.363701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:59.367032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4dd7b-06c4-447a-9c28-e84e5f318329 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:59.410194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9844e-374c-42ec-9d1a-c913cf69ca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:59.411009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9844e-374c-42ec-9d1a-c913cf69ca5c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:59.411295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9844e-374c-42ec-9d1a-c913cf69ca5c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:59.411625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9844e-374c-42ec-9d1a-c913cf69ca5c None admin] PATH_INFO: `/v3/users/f3c4ee9213164a87a6278f251918bdd8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:59.428000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9844e-374c-42ec-9d1a-c913cf69ca5c None admin] RBAC: Authorizing `identity:get_user(user_id=f3c4ee9213164a87a6278f251918bdd8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:59.429535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9844e-374c-42ec-9d1a-c913cf69ca5c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:59.431382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 102/195] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:59 2022] GET /identity/v3/users/f3c4ee9213164a87a6278f251918bdd8 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:59.432284 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9844e-374c-42ec-9d1a-c913cf69ca5c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:59.466970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb3e7f-d483-49b6-97be-fba5626b64f1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:59.492869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:59.516414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197bee0-4aa8-430d-93e5-c2198a10ebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:59.517873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197bee0-4aa8-430d-93e5-c2198a10ebd1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:59.518809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197bee0-4aa8-430d-93e5-c2198a10ebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:59.518809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197bee0-4aa8-430d-93e5-c2198a10ebd1 None admin] PATH_INFO: `/v3/users/3003cc1d2f22405cbb7a460726ac432a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:59.520711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197bee0-4aa8-430d-93e5-c2198a10ebd1 None admin] RBAC: Authorizing `identity:get_user(user_id=3003cc1d2f22405cbb7a460726ac432a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:59.522584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197bee0-4aa8-430d-93e5-c2198a10ebd1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:59.525340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 94/196] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:05:59 2022] GET /identity/v3/users/3003cc1d2f22405cbb7a460726ac432a => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:59.526532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197bee0-4aa8-430d-93e5-c2198a10ebd1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:59.580117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9844e-374c-42ec-9d1a-c913cf69ca5c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:59.622106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27371e4-dcc2-4ebb-b1c9-d97ea792a55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:59.623350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27371e4-dcc2-4ebb-b1c9-d97ea792a55e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:59.623350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27371e4-dcc2-4ebb-b1c9-d97ea792a55e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:59.623350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27371e4-dcc2-4ebb-b1c9-d97ea792a55e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:59.640219 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27371e4-dcc2-4ebb-b1c9-d97ea792a55e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d27371e4-dcc2-4ebb-b1c9-d97ea792a55e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:05:59.649268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:05:59.650236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:05:59.651018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:05:59.651643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 103/197] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:05:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:05:59.652462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27371e4-dcc2-4ebb-b1c9-d97ea792a55e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:59.655689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:59.669272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197bee0-4aa8-430d-93e5-c2198a10ebd1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:59.712512 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413dc0e-0136-49c8-9b21-de78fa8f2b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:59.713187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413dc0e-0136-49c8-9b21-de78fa8f2b18 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:59.713479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413dc0e-0136-49c8-9b21-de78fa8f2b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:59.713791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413dc0e-0136-49c8-9b21-de78fa8f2b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:59.784925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27371e4-dcc2-4ebb-b1c9-d97ea792a55e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:59.794344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:59.801282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413dc0e-0136-49c8-9b21-de78fa8f2b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:59.803427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413dc0e-0136-49c8-9b21-de78fa8f2b18 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:59.829134 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa81b8-cbf0-4834-9ca7-8638516b0424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:59.830324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa81b8-cbf0-4834-9ca7-8638516b0424 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:59.830613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa81b8-cbf0-4834-9ca7-8638516b0424 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:59.830892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa81b8-cbf0-4834-9ca7-8638516b0424 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:59.921751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa81b8-cbf0-4834-9ca7-8638516b0424 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:59.922501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:59.923683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa81b8-cbf0-4834-9ca7-8638516b0424 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:59.935439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 95/198] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:05:59.937599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413dc0e-0136-49c8-9b21-de78fa8f2b18 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:00.040326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 104/199] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:05:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:00.041003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa81b8-cbf0-4834-9ca7-8638516b0424 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:00.051989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:00.073503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413dc0e-0136-49c8-9b21-de78fa8f2b18 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:00.117932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b60e4c-f476-4176-b41f-1ef8439b3f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:00.119048 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b60e4c-f476-4176-b41f-1ef8439b3f59 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:00.119581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b60e4c-f476-4176-b41f-1ef8439b3f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:00.120111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b60e4c-f476-4176-b41f-1ef8439b3f59 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:00.122058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b60e4c-f476-4176-b41f-1ef8439b3f59 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:00.122635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b60e4c-f476-4176-b41f-1ef8439b3f59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:00.125117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b60e4c-f476-4176-b41f-1ef8439b3f59 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:00.137449 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 96/200] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:05:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:00.138918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b60e4c-f476-4176-b41f-1ef8439b3f59 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:00.146845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa81b8-cbf0-4834-9ca7-8638516b0424 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:00.182984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9294b8a-7879-4a01-9fd9-acb2a6dda673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:00.183766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9294b8a-7879-4a01-9fd9-acb2a6dda673 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:00.184047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9294b8a-7879-4a01-9fd9-acb2a6dda673 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:00.184319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9294b8a-7879-4a01-9fd9-acb2a6dda673 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:00.185501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9294b8a-7879-4a01-9fd9-acb2a6dda673 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:00.187936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9294b8a-7879-4a01-9fd9-acb2a6dda673 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:00.189786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 105/201] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:00 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:00.190569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9294b8a-7879-4a01-9fd9-acb2a6dda673 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:00.212032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:00.270225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:00.272218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b60e4c-f476-4176-b41f-1ef8439b3f59 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:00.298628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9294b8a-7879-4a01-9fd9-acb2a6dda673 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:00.317659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83b0bd-9a65-49ae-8390-bab21978e240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:00.318879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83b0bd-9a65-49ae-8390-bab21978e240 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:00.319348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83b0bd-9a65-49ae-8390-bab21978e240 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:00.319925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83b0bd-9a65-49ae-8390-bab21978e240 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:00.322782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83b0bd-9a65-49ae-8390-bab21978e240 None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:00.324881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83b0bd-9a65-49ae-8390-bab21978e240 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:00.327770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 97/202] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:00 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:00.329517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83b0bd-9a65-49ae-8390-bab21978e240 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:00.336676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b7531-dc5a-4a1b-b7c2-f2f9be207d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:00.337558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b7531-dc5a-4a1b-b7c2-f2f9be207d32 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:00.337864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b7531-dc5a-4a1b-b7c2-f2f9be207d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:00.338136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b7531-dc5a-4a1b-b7c2-f2f9be207d32 None admin] PATH_INFO: `/v3/users/f3c4ee9213164a87a6278f251918bdd8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:00.339334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b7531-dc5a-4a1b-b7c2-f2f9be207d32 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c4ee9213164a87a6278f251918bdd8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:00.340781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b7531-dc5a-4a1b-b7c2-f2f9be207d32 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:00.342979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 106/203] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:00 2022] GET /identity/v3/users/f3c4ee9213164a87a6278f251918bdd8 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:00.344172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b7531-dc5a-4a1b-b7c2-f2f9be207d32 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:00.448212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83b0bd-9a65-49ae-8390-bab21978e240 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:00.458562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b7531-dc5a-4a1b-b7c2-f2f9be207d32 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:00.472606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:00.487124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17f77c-7871-4ae9-b728-871008b43d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:00.487794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17f77c-7871-4ae9-b728-871008b43d58 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:00.488129 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17f77c-7871-4ae9-b728-871008b43d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:00.488414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17f77c-7871-4ae9-b728-871008b43d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:00.489247 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17f77c-7871-4ae9-b728-871008b43d58 None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=affcd8ec317b49faaaa94fdb779a9948, user.id=f3c4ee9213164a87a6278f251918bdd8) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:00.489561 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17f77c-7871-4ae9-b728-871008b43d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:00.491053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17f77c-7871-4ae9-b728-871008b43d58 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:00.495242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e561562-e6bc-47c7-b2e9-28b0fb5d14d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:00.495899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e561562-e6bc-47c7-b2e9-28b0fb5d14d7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:00.496180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e561562-e6bc-47c7-b2e9-28b0fb5d14d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:00.496453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e561562-e6bc-47c7-b2e9-28b0fb5d14d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:00.497286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e561562-e6bc-47c7-b2e9-28b0fb5d14d7 None admin] RBAC: Adding query filter params (role.id=a200b6cb596344e385e5f234488c6018, scope.project.id=82df840b0bc44187a1a9db5c2a668847, user.id=3003cc1d2f22405cbb7a460726ac432a) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:00.497516 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 98/204] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:00 2022] GET /identity/v3/role_assignments?user.id=f3c4ee9213164a87a6278f251918bdd8&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:00.497666 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e561562-e6bc-47c7-b2e9-28b0fb5d14d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:00.498283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17f77c-7871-4ae9-b728-871008b43d58 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:00.500300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e561562-e6bc-47c7-b2e9-28b0fb5d14d7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:00.507212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 107/205] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:00 2022] GET /identity/v3/role_assignments?user.id=3003cc1d2f22405cbb7a460726ac432a&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=a200b6cb596344e385e5f234488c6018 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:00.643432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17f77c-7871-4ae9-b728-871008b43d58 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:00.700490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021e663-d035-4ef6-9e45-84e73ed44378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:00.701270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021e663-d035-4ef6-9e45-84e73ed44378 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:00.701445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021e663-d035-4ef6-9e45-84e73ed44378 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:00.702175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021e663-d035-4ef6-9e45-84e73ed44378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:00.793642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:00.812094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021e663-d035-4ef6-9e45-84e73ed44378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:00.814101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021e663-d035-4ef6-9e45-84e73ed44378 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:00.928777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 99/206] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:00.934477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e561562-e6bc-47c7-b2e9-28b0fb5d14d7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:00.945459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:01.054898 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e561562-e6bc-47c7-b2e9-28b0fb5d14d7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:01.096140 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6babc-935e-4edd-969a-f55b92d3d477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:01.097392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6babc-935e-4edd-969a-f55b92d3d477 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:01.097904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6babc-935e-4edd-969a-f55b92d3d477 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:01.098380 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6babc-935e-4edd-969a-f55b92d3d477 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:01.099783 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6babc-935e-4edd-969a-f55b92d3d477 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:01.101447 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6babc-935e-4edd-969a-f55b92d3d477 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:01.103395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 108/207] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:00 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:01.109514 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021e663-d035-4ef6-9e45-84e73ed44378 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:01.208930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:01.224467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021e663-d035-4ef6-9e45-84e73ed44378 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:01.263733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d668bf-bed5-47d3-b22e-9438c71c8eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:01.264792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d668bf-bed5-47d3-b22e-9438c71c8eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:01.265308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d668bf-bed5-47d3-b22e-9438c71c8eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:01.265823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d668bf-bed5-47d3-b22e-9438c71c8eb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:01.266852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d668bf-bed5-47d3-b22e-9438c71c8eb6 None admin] RBAC: Adding query filter params (role.id=e7180b44fc2945568dfb83dfaf8b5c79, scope.project.id=affcd8ec317b49faaaa94fdb779a9948, user.id=f3c4ee9213164a87a6278f251918bdd8) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:01.267370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d668bf-bed5-47d3-b22e-9438c71c8eb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:01.269076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d668bf-bed5-47d3-b22e-9438c71c8eb6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:01.275683 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 100/208] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:01 2022] GET /identity/v3/role_assignments?user.id=f3c4ee9213164a87a6278f251918bdd8&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=e7180b44fc2945568dfb83dfaf8b5c79 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:01.864802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80219d-184b-4d8d-88eb-4b3dc488adf4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:01.865090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80219d-184b-4d8d-88eb-4b3dc488adf4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:01.865537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80219d-184b-4d8d-88eb-4b3dc488adf4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:01.867572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 109/209] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:01.873697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4476-1804-4004-b71a-c4b0d1b7eaea None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:01.874684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4476-1804-4004-b71a-c4b0d1b7eaea None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:01.875165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4476-1804-4004-b71a-c4b0d1b7eaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:01.896172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-53ca4476-1804-4004-b71a-c4b0d1b7eaea None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:01.897756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-53ca4476-1804-4004-b71a-c4b0d1b7eaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:01.949299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67dfc5-7b8c-4e3e-9c79-37ed3400ac95 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:01.950279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67dfc5-7b8c-4e3e-9c79-37ed3400ac95 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:01.950754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67dfc5-7b8c-4e3e-9c79-37ed3400ac95 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:01.957352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 110/210] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:01.959382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16286536-8b58-48d8-a31c-80f7feafd25a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:01.959878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16286536-8b58-48d8-a31c-80f7feafd25a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:01.960427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16286536-8b58-48d8-a31c-80f7feafd25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:01.971759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 101/211] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:01.980125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-16286536-8b58-48d8-a31c-80f7feafd25a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:01.981593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-16286536-8b58-48d8-a31c-80f7feafd25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:01.998358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca4476-1804-4004-b71a-c4b0d1b7eaea None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:02.061854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 111/212] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:06:02.083348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16286536-8b58-48d8-a31c-80f7feafd25a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:02.087083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:02.089236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:02.113750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca4476-1804-4004-b71a-c4b0d1b7eaea None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:02.149640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240731ff-1f3e-40e7-b4c7-bcc127d09136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:02.150409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240731ff-1f3e-40e7-b4c7-bcc127d09136 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:02.150699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240731ff-1f3e-40e7-b4c7-bcc127d09136 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:02.150982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240731ff-1f3e-40e7-b4c7-bcc127d09136 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:02.162419 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240731ff-1f3e-40e7-b4c7-bcc127d09136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-240731ff-1f3e-40e7-b4c7-bcc127d09136 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:02.170997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:02.172068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:02.172906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:02.174107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 102/213] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:02.181806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240731ff-1f3e-40e7-b4c7-bcc127d09136 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:02.213697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16286536-8b58-48d8-a31c-80f7feafd25a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:02.251367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30442d-f48a-4007-9ffa-d95185af6329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:02.252204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30442d-f48a-4007-9ffa-d95185af6329 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:02.252558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30442d-f48a-4007-9ffa-d95185af6329 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:02.252835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30442d-f48a-4007-9ffa-d95185af6329 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:02.254041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30442d-f48a-4007-9ffa-d95185af6329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:02.255683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30442d-f48a-4007-9ffa-d95185af6329 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:02.257412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 112/214] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:02.266849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30442d-f48a-4007-9ffa-d95185af6329 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:02.288568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240731ff-1f3e-40e7-b4c7-bcc127d09136 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:02.324241 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77e5fc-8dc3-4831-a559-bce83e69110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:02.324924 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77e5fc-8dc3-4831-a559-bce83e69110d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:02.325210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77e5fc-8dc3-4831-a559-bce83e69110d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:02.325487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77e5fc-8dc3-4831-a559-bce83e69110d None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:02.326287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77e5fc-8dc3-4831-a559-bce83e69110d None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:02.326732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77e5fc-8dc3-4831-a559-bce83e69110d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:02.328441 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77e5fc-8dc3-4831-a559-bce83e69110d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:02.337205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 103/215] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:02.343821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77e5fc-8dc3-4831-a559-bce83e69110d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:02.347950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:02.349914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:02.385110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30442d-f48a-4007-9ffa-d95185af6329 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:02.424672 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50742e2-7c79-443c-af0e-61a2992d0109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:02.425818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50742e2-7c79-443c-af0e-61a2992d0109 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:02.426351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50742e2-7c79-443c-af0e-61a2992d0109 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:02.426843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50742e2-7c79-443c-af0e-61a2992d0109 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:02.427887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50742e2-7c79-443c-af0e-61a2992d0109 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:02.429778 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50742e2-7c79-443c-af0e-61a2992d0109 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:02.455532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77e5fc-8dc3-4831-a559-bce83e69110d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:02.465145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 113/216] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:06:02 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:06:02.500382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634d2bc-37ca-4cde-9c9d-1423716930d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:02.501356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634d2bc-37ca-4cde-9c9d-1423716930d3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:02.501696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634d2bc-37ca-4cde-9c9d-1423716930d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:02.502251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634d2bc-37ca-4cde-9c9d-1423716930d3 None admin] PATH_INFO: `/v3/users/051466d6c63643fc83a5f8e1b0809fde` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:02.520320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634d2bc-37ca-4cde-9c9d-1423716930d3 None admin] RBAC: Authorizing `identity:get_user(user_id=051466d6c63643fc83a5f8e1b0809fde)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:02.522031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634d2bc-37ca-4cde-9c9d-1423716930d3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:02.524096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 104/217] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:02 2022] GET /identity/v3/users/051466d6c63643fc83a5f8e1b0809fde => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:02.529741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50742e2-7c79-443c-af0e-61a2992d0109 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:02.650315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50742e2-7c79-443c-af0e-61a2992d0109 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:02.680321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:02.689629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d075683-71f6-45bb-b28e-aee06cc408d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:02.691426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d075683-71f6-45bb-b28e-aee06cc408d3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:02.691709 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d075683-71f6-45bb-b28e-aee06cc408d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:02.691989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d075683-71f6-45bb-b28e-aee06cc408d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:02.767651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d075683-71f6-45bb-b28e-aee06cc408d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:02.769180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d075683-71f6-45bb-b28e-aee06cc408d3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:02.882075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 114/218] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:02.887413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634d2bc-37ca-4cde-9c9d-1423716930d3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:02.963083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:03.008117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634d2bc-37ca-4cde-9c9d-1423716930d3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:03.034861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bae3d-4a0f-433f-9ac0-dbfbaa4c3458 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:03.035392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bae3d-4a0f-433f-9ac0-dbfbaa4c3458 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:03.035392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bae3d-4a0f-433f-9ac0-dbfbaa4c3458 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:03.037003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 115/219] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:03.042030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f59505-ad71-4108-beb5-a89e6a3ec55b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:03.042318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f59505-ad71-4108-beb5-a89e6a3ec55b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:03.042601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f59505-ad71-4108-beb5-a89e6a3ec55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:03.044758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af146676-e295-41a3-9129-d4210b4dc5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:03.045984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af146676-e295-41a3-9129-d4210b4dc5ff None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:03.046497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af146676-e295-41a3-9129-d4210b4dc5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:03.046802 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af146676-e295-41a3-9129-d4210b4dc5ff None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:03.048015 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af146676-e295-41a3-9129-d4210b4dc5ff None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:03.049470 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af146676-e295-41a3-9129-d4210b4dc5ff None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:03.051506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 105/220] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:02 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:03.057097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af146676-e295-41a3-9129-d4210b4dc5ff None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:03.061481 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-97f59505-ad71-4108-beb5-a89e6a3ec55b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:03.062821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-97f59505-ad71-4108-beb5-a89e6a3ec55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:03.090072 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:03.172152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 116/221] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:06:03.222529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af146676-e295-41a3-9129-d4210b4dc5ff None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:03.229706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f59505-ad71-4108-beb5-a89e6a3ec55b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:03.279458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:03.297609 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cd8f5-e194-46ca-b0ed-15fe4596d52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:03.298312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd8f5-e194-46ca-b0ed-15fe4596d52a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:03.298597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd8f5-e194-46ca-b0ed-15fe4596d52a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:03.298876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd8f5-e194-46ca-b0ed-15fe4596d52a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:03.299713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cd8f5-e194-46ca-b0ed-15fe4596d52a None admin] RBAC: Adding query filter params (role.id=7f6dd280cf8941dcb8f79ecf88dbda78, scope.project.id=affcd8ec317b49faaaa94fdb779a9948, user.id=051466d6c63643fc83a5f8e1b0809fde) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:03.300025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cd8f5-e194-46ca-b0ed-15fe4596d52a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:03.301477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cd8f5-e194-46ca-b0ed-15fe4596d52a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:03.319161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 106/222] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:03 2022] GET /identity/v3/role_assignments?user.id=051466d6c63643fc83a5f8e1b0809fde&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:03.378988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f59505-ad71-4108-beb5-a89e6a3ec55b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:03.436567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dfe2c-7dec-4aaa-b25a-0467828dbe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:03.437433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dfe2c-7dec-4aaa-b25a-0467828dbe8a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:03.437731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dfe2c-7dec-4aaa-b25a-0467828dbe8a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:03.438014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dfe2c-7dec-4aaa-b25a-0467828dbe8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:03.459889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4dfe2c-7dec-4aaa-b25a-0467828dbe8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5b4dfe2c-7dec-4aaa-b25a-0467828dbe8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:03.467679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:03.468746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:03.470632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 117/223] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:06:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:03.478507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cd8f5-e194-46ca-b0ed-15fe4596d52a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:03.653752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cd8f5-e194-46ca-b0ed-15fe4596d52a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:03.690743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0553a5-d939-4523-a2f8-043d4f69b719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:03.691432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0553a5-d939-4523-a2f8-043d4f69b719 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:03.691713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0553a5-d939-4523-a2f8-043d4f69b719 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:03.691985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0553a5-d939-4523-a2f8-043d4f69b719 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:03.692818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0553a5-d939-4523-a2f8-043d4f69b719 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:03.693118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0553a5-d939-4523-a2f8-043d4f69b719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:03.694560 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0553a5-d939-4523-a2f8-043d4f69b719 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:03.703911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 107/224] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:03.709517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dfe2c-7dec-4aaa-b25a-0467828dbe8a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:03.758498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:03.769648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909b0d-f820-44e5-8ee3-56ea98b380c2 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:03.770013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909b0d-f820-44e5-8ee3-56ea98b380c2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:03.770290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909b0d-f820-44e5-8ee3-56ea98b380c2 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:03.773395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 108/225] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:03.779183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a78b43-b5c9-4d49-8f9b-8f7f0f804435 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:03.779471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a78b43-b5c9-4d49-8f9b-8f7f0f804435 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:03.780283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a78b43-b5c9-4d49-8f9b-8f7f0f804435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:03.830754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e2a78b43-b5c9-4d49-8f9b-8f7f0f804435 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:03.831954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e2a78b43-b5c9-4d49-8f9b-8f7f0f804435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:03.858645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dfe2c-7dec-4aaa-b25a-0467828dbe8a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:03.894104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdebf93-20b5-4d7a-b9a6-e60e93e82ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:03.894988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdebf93-20b5-4d7a-b9a6-e60e93e82ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:03.895273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdebf93-20b5-4d7a-b9a6-e60e93e82ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:03.895546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdebf93-20b5-4d7a-b9a6-e60e93e82ea8 None admin] PATH_INFO: `/v3/users/f3c4ee9213164a87a6278f251918bdd8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:03.896735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdebf93-20b5-4d7a-b9a6-e60e93e82ea8 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c4ee9213164a87a6278f251918bdd8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:03.898304 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdebf93-20b5-4d7a-b9a6-e60e93e82ea8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:03.899998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 118/226] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:03 2022] GET /identity/v3/users/f3c4ee9213164a87a6278f251918bdd8 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:03.907503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdebf93-20b5-4d7a-b9a6-e60e93e82ea8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:03.975862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:04.003428 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:04.020967 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdebf93-20b5-4d7a-b9a6-e60e93e82ea8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:04.023559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 109/227] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:04.058988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a78b43-b5c9-4d49-8f9b-8f7f0f804435 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:04.059193 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb340de-2c25-4b44-90bc-9050f061a1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:04.059893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb340de-2c25-4b44-90bc-9050f061a1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:04.060176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb340de-2c25-4b44-90bc-9050f061a1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:04.060451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb340de-2c25-4b44-90bc-9050f061a1e7 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:04.062074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb340de-2c25-4b44-90bc-9050f061a1e7 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:04.065855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb340de-2c25-4b44-90bc-9050f061a1e7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:04.069119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 119/228] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:03 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:04.071534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb340de-2c25-4b44-90bc-9050f061a1e7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:04.191478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb340de-2c25-4b44-90bc-9050f061a1e7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:04.233328 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304469a-25b0-414f-9380-97dd3e61c2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:04.234656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304469a-25b0-414f-9380-97dd3e61c2d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:04.234939 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304469a-25b0-414f-9380-97dd3e61c2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:04.235215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304469a-25b0-414f-9380-97dd3e61c2d5 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948/users/f3c4ee9213164a87a6278f251918bdd8/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:04.237291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304469a-25b0-414f-9380-97dd3e61c2d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=affcd8ec317b49faaaa94fdb779a9948, user_id=f3c4ee9213164a87a6278f251918bdd8, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:04.238798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304469a-25b0-414f-9380-97dd3e61c2d5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:04.256059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 120/229] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:06:04 2022] PUT /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948/users/f3c4ee9213164a87a6278f251918bdd8/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:06:04.317693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a78b43-b5c9-4d49-8f9b-8f7f0f804435 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:04.369766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d5de4-95b7-4134-8894-4c056e48879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:04.370620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d5de4-95b7-4134-8894-4c056e48879e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:04.370909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d5de4-95b7-4134-8894-4c056e48879e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:04.371189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d5de4-95b7-4134-8894-4c056e48879e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:04.387354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d5de4-95b7-4134-8894-4c056e48879e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d60d5de4-95b7-4134-8894-4c056e48879e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:04.400269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:04.401220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:04.403622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 110/230] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:06:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:04.414898 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304469a-25b0-414f-9380-97dd3e61c2d5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:04.455046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:04.568916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304469a-25b0-414f-9380-97dd3e61c2d5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:04.606365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fad78-a868-49b2-8b1f-38b42d96b697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:04.607038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fad78-a868-49b2-8b1f-38b42d96b697 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:04.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fad78-a868-49b2-8b1f-38b42d96b697 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:04.607598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fad78-a868-49b2-8b1f-38b42d96b697 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:04.608718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fad78-a868-49b2-8b1f-38b42d96b697 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:04.609117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fad78-a868-49b2-8b1f-38b42d96b697 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:04.610596 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fad78-a868-49b2-8b1f-38b42d96b697 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:04.619381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 121/231] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:04.625547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d5de4-95b7-4134-8894-4c056e48879e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:04.675077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:04.735290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d5de4-95b7-4134-8894-4c056e48879e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:04.784795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ffc43-a15b-4205-ae74-8aef1b8a5054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:04.785688 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ffc43-a15b-4205-ae74-8aef1b8a5054 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:04.785994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ffc43-a15b-4205-ae74-8aef1b8a5054 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:04.786267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ffc43-a15b-4205-ae74-8aef1b8a5054 None admin] PATH_INFO: `/v3/users/f3c4ee9213164a87a6278f251918bdd8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:04.787553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ffc43-a15b-4205-ae74-8aef1b8a5054 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c4ee9213164a87a6278f251918bdd8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:04.789003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ffc43-a15b-4205-ae74-8aef1b8a5054 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:04.790669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 111/232] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:04 2022] GET /identity/v3/users/f3c4ee9213164a87a6278f251918bdd8 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:04.796641 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fad78-a868-49b2-8b1f-38b42d96b697 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:04.893475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:04.923156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fad78-a868-49b2-8b1f-38b42d96b697 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:04.969153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09144d29-5f1b-4410-9c01-0c464a038435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:04.970061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09144d29-5f1b-4410-9c01-0c464a038435 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:04.970346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09144d29-5f1b-4410-9c01-0c464a038435 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:04.970619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09144d29-5f1b-4410-9c01-0c464a038435 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:04.971835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09144d29-5f1b-4410-9c01-0c464a038435 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:04.973300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09144d29-5f1b-4410-9c01-0c464a038435 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:04.975255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 122/233] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:04 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:04.980799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ffc43-a15b-4205-ae74-8aef1b8a5054 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:05.101314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ffc43-a15b-4205-ae74-8aef1b8a5054 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:05.115302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:05.138220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db8b32-8093-43d2-ab7b-58feddd680da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:05.139680 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db8b32-8093-43d2-ab7b-58feddd680da None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:05.139974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db8b32-8093-43d2-ab7b-58feddd680da None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:05.140247 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db8b32-8093-43d2-ab7b-58feddd680da None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948/users/f3c4ee9213164a87a6278f251918bdd8/roles/e7180b44fc2945568dfb83dfaf8b5c79` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:05.142784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db8b32-8093-43d2-ab7b-58feddd680da None admin] RBAC: Authorizing `identity:create_grant(project_id=affcd8ec317b49faaaa94fdb779a9948, user_id=f3c4ee9213164a87a6278f251918bdd8, role_id=e7180b44fc2945568dfb83dfaf8b5c79)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:05.144303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db8b32-8093-43d2-ab7b-58feddd680da None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:05.160934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 112/234] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:06:04 2022] PUT /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948/users/f3c4ee9213164a87a6278f251918bdd8/roles/e7180b44fc2945568dfb83dfaf8b5c79 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:06:05.401459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cccd0d-4e70-4368-9bed-8539516f77a0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:05.401982 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cccd0d-4e70-4368-9bed-8539516f77a0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:05.401982 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cccd0d-4e70-4368-9bed-8539516f77a0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:05.403642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 123/235] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:05.415916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da78872-e08b-4f18-ac2a-33fe0326be8e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:05.416235 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da78872-e08b-4f18-ac2a-33fe0326be8e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:05.416502 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da78872-e08b-4f18-ac2a-33fe0326be8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:05.438939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1da78872-e08b-4f18-ac2a-33fe0326be8e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:05.440182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1da78872-e08b-4f18-ac2a-33fe0326be8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:05.514216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 113/236] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:05.537506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cccd0d-4e70-4368-9bed-8539516f77a0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:05.655427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cccd0d-4e70-4368-9bed-8539516f77a0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:05.671524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f8b59-c9e1-442e-bc58-595701af67ef None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:05.671813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f8b59-c9e1-442e-bc58-595701af67ef None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:05.672089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f8b59-c9e1-442e-bc58-595701af67ef None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:05.673657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 114/237] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:05.678920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa5ef6-8151-43b0-9fb1-6f711678f0f6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:05.679206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa5ef6-8151-43b0-9fb1-6f711678f0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:05.679491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa5ef6-8151-43b0-9fb1-6f711678f0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:05.705540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9daa5ef6-8151-43b0-9fb1-6f711678f0f6 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:05.706762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9daa5ef6-8151-43b0-9fb1-6f711678f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:05.733856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e28b3d-e2d2-45e1-b393-c7ffa464ae1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:05.734724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e28b3d-e2d2-45e1-b393-c7ffa464ae1b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:05.735009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e28b3d-e2d2-45e1-b393-c7ffa464ae1b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:05.735283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e28b3d-e2d2-45e1-b393-c7ffa464ae1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:05.761182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e28b3d-e2d2-45e1-b393-c7ffa464ae1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-02e28b3d-e2d2-45e1-b393-c7ffa464ae1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:05.776014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:05.776950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:05.777799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 124/238] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:05.799445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e28b3d-e2d2-45e1-b393-c7ffa464ae1b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:05.810094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:05.855350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 115/239] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:06:05.903906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa5ef6-8151-43b0-9fb1-6f711678f0f6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:05.946559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:05.970489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e28b3d-e2d2-45e1-b393-c7ffa464ae1b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:06.027064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa5ef6-8151-43b0-9fb1-6f711678f0f6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:06.046553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc3618-4a3e-4943-874e-b49b9f4fab67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:06.047233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc3618-4a3e-4943-874e-b49b9f4fab67 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:06.047519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc3618-4a3e-4943-874e-b49b9f4fab67 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:06.047797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc3618-4a3e-4943-874e-b49b9f4fab67 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:06.048562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc3618-4a3e-4943-874e-b49b9f4fab67 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:06.048872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc3618-4a3e-4943-874e-b49b9f4fab67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:06.050313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc3618-4a3e-4943-874e-b49b9f4fab67 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:06.070611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 125/240] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:06.080013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc3618-4a3e-4943-874e-b49b9f4fab67 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:06.103884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f405-8f32-48b2-96ae-5d464933601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:06.105154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f405-8f32-48b2-96ae-5d464933601f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:06.105645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f405-8f32-48b2-96ae-5d464933601f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:06.106145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f405-8f32-48b2-96ae-5d464933601f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:06.110538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314f405-8f32-48b2-96ae-5d464933601f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:06.112268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314f405-8f32-48b2-96ae-5d464933601f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:06.114501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 116/241] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:06.121492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f405-8f32-48b2-96ae-5d464933601f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:06.180708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:06.238895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc3618-4a3e-4943-874e-b49b9f4fab67 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:06.303497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793dac1-d109-4e55-80a6-fa5f7c4014a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:06.304378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793dac1-d109-4e55-80a6-fa5f7c4014a5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:06.304661 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793dac1-d109-4e55-80a6-fa5f7c4014a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:06.304975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793dac1-d109-4e55-80a6-fa5f7c4014a5 None admin] PATH_INFO: `/v3/users/5ebdc65cf1db44c4b88fc4125e13e39a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:06.324947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:06.326229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793dac1-d109-4e55-80a6-fa5f7c4014a5 None admin] RBAC: Authorizing `identity:get_user(user_id=5ebdc65cf1db44c4b88fc4125e13e39a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:06.327812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793dac1-d109-4e55-80a6-fa5f7c4014a5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:06.329757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 126/242] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:06 2022] GET /identity/v3/users/5ebdc65cf1db44c4b88fc4125e13e39a => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:06.330984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a418cb-fed8-4911-a70d-42fae15ed244 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:06.331274 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a418cb-fed8-4911-a70d-42fae15ed244 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:06.331551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a418cb-fed8-4911-a70d-42fae15ed244 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:06.332925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 127/243] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:06.338482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424d64-0c4d-42af-b53e-63d2337107b9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:06.338777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424d64-0c4d-42af-b53e-63d2337107b9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:06.339093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424d64-0c4d-42af-b53e-63d2337107b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:06.344724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f405-8f32-48b2-96ae-5d464933601f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:06.360788 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6e424d64-0c4d-42af-b53e-63d2337107b9 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:06.362072 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6e424d64-0c4d-42af-b53e-63d2337107b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:06.386408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa581e-2f46-4350-8c6f-be64224bbdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:06.387097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa581e-2f46-4350-8c6f-be64224bbdfd None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:06.387377 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa581e-2f46-4350-8c6f-be64224bbdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:06.387654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa581e-2f46-4350-8c6f-be64224bbdfd None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:06.388369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa581e-2f46-4350-8c6f-be64224bbdfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:06.391093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa581e-2f46-4350-8c6f-be64224bbdfd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:06.417809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 117/244] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:06:06 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:06:06.418820 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa581e-2f46-4350-8c6f-be64224bbdfd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:06.470165 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 128/245] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:06.494471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e424d64-0c4d-42af-b53e-63d2337107b9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:06.600944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:06.627193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa581e-2f46-4350-8c6f-be64224bbdfd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:06.664347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9fd97-199d-4cbf-a71e-6d7b9df480a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:06.665066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9fd97-199d-4cbf-a71e-6d7b9df480a0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:06.665363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9fd97-199d-4cbf-a71e-6d7b9df480a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:06.665643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9fd97-199d-4cbf-a71e-6d7b9df480a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:06.693248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:06.693668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e424d64-0c4d-42af-b53e-63d2337107b9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:06.738010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7f21d-fd8d-4207-b9e7-2a4fae9fb4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:06.739175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7f21d-fd8d-4207-b9e7-2a4fae9fb4fe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:06.739652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7f21d-fd8d-4207-b9e7-2a4fae9fb4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:06.740159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7f21d-fd8d-4207-b9e7-2a4fae9fb4fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:06.745091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9fd97-199d-4cbf-a71e-6d7b9df480a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:06.746572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9fd97-199d-4cbf-a71e-6d7b9df480a0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:06.752880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7f21d-fd8d-4207-b9e7-2a4fae9fb4fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6cc7f21d-fd8d-4207-b9e7-2a4fae9fb4fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:06.762032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:06.762957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:06.763779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:06.766813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 129/246] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:06.780176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7f21d-fd8d-4207-b9e7-2a4fae9fb4fe None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:06.861908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 118/247] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:06.867515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9fd97-199d-4cbf-a71e-6d7b9df480a0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:06.896643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7f21d-fd8d-4207-b9e7-2a4fae9fb4fe None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:06.915488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:06.927772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:06.940193 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c3d89-e464-4190-8cac-b462ce2dec30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:06.941023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c3d89-e464-4190-8cac-b462ce2dec30 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:06.941405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c3d89-e464-4190-8cac-b462ce2dec30 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:06.941855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c3d89-e464-4190-8cac-b462ce2dec30 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:06.942708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c3d89-e464-4190-8cac-b462ce2dec30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:06.943145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c3d89-e464-4190-8cac-b462ce2dec30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:06.944884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c3d89-e464-4190-8cac-b462ce2dec30 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:06.956252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 130/248] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:06.962189 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c3d89-e464-4190-8cac-b462ce2dec30 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:06.981262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9fd97-199d-4cbf-a71e-6d7b9df480a0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:07.018269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb76b32-e1e4-482a-af20-7cda8cd33a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:07.019360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb76b32-e1e4-482a-af20-7cda8cd33a86 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:07.019789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb76b32-e1e4-482a-af20-7cda8cd33a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:07.020210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb76b32-e1e4-482a-af20-7cda8cd33a86 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:07.021619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb76b32-e1e4-482a-af20-7cda8cd33a86 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:07.023276 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb76b32-e1e4-482a-af20-7cda8cd33a86 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:07.025472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 119/249] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:06 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:07.031925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb76b32-e1e4-482a-af20-7cda8cd33a86 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:07.084137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c3d89-e464-4190-8cac-b462ce2dec30 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:07.120997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abdd56-0cfc-4dae-b228-01338bc3fd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:07.122508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abdd56-0cfc-4dae-b228-01338bc3fd01 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:07.122508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abdd56-0cfc-4dae-b228-01338bc3fd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:07.122775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abdd56-0cfc-4dae-b228-01338bc3fd01 None admin] PATH_INFO: `/v3/users/051466d6c63643fc83a5f8e1b0809fde` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:07.124681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abdd56-0cfc-4dae-b228-01338bc3fd01 None admin] RBAC: Authorizing `identity:get_user(user_id=051466d6c63643fc83a5f8e1b0809fde)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:07.126374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abdd56-0cfc-4dae-b228-01338bc3fd01 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:07.128403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 131/250] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:06 2022] GET /identity/v3/users/051466d6c63643fc83a5f8e1b0809fde => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:07.133903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abdd56-0cfc-4dae-b228-01338bc3fd01 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:07.143210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb76b32-e1e4-482a-af20-7cda8cd33a86 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:07.178099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:07.182556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d67be-2487-4c89-8b75-f28023ff97bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:07.183434 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d67be-2487-4c89-8b75-f28023ff97bd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:07.183864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d67be-2487-4c89-8b75-f28023ff97bd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:07.184303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d67be-2487-4c89-8b75-f28023ff97bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:07.185301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d67be-2487-4c89-8b75-f28023ff97bd None admin] RBAC: Adding query filter params (role.id=a200b6cb596344e385e5f234488c6018, scope.project.id=affcd8ec317b49faaaa94fdb779a9948, user.id=5ebdc65cf1db44c4b88fc4125e13e39a) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:07.185891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d67be-2487-4c89-8b75-f28023ff97bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:07.188027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d67be-2487-4c89-8b75-f28023ff97bd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:07.188627 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:07.195409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 120/251] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:07 2022] GET /identity/v3/role_assignments?user.id=5ebdc65cf1db44c4b88fc4125e13e39a&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=a200b6cb596344e385e5f234488c6018 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:07.196952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe88f7-f9be-489f-9ef4-cfb90e5f4517 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:07.197389 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe88f7-f9be-489f-9ef4-cfb90e5f4517 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:07.202079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe88f7-f9be-489f-9ef4-cfb90e5f4517 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:07.203822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 121/252] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:07.211400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571c682-d963-4ff4-a330-65493ae8467c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:07.211913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571c682-d963-4ff4-a330-65493ae8467c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:07.212361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571c682-d963-4ff4-a330-65493ae8467c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:07.246858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9571c682-d963-4ff4-a330-65493ae8467c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:07.248391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9571c682-d963-4ff4-a330-65493ae8467c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:07.269432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abdd56-0cfc-4dae-b228-01338bc3fd01 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:07.323372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5ef43-5660-4f4a-89ce-5b5ad9eb3f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:07.324356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5ef43-5660-4f4a-89ce-5b5ad9eb3f96 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:07.324713 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5ef43-5660-4f4a-89ce-5b5ad9eb3f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:07.325059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5ef43-5660-4f4a-89ce-5b5ad9eb3f96 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:07.327895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5ef43-5660-4f4a-89ce-5b5ad9eb3f96 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:07.329489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5ef43-5660-4f4a-89ce-5b5ad9eb3f96 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:07.332790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 132/253] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:07 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:07.338165 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5ef43-5660-4f4a-89ce-5b5ad9eb3f96 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:07.353009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 122/254] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:06:07.379107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571c682-d963-4ff4-a330-65493ae8467c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:07.462199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5ef43-5660-4f4a-89ce-5b5ad9eb3f96 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:07.491085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:07.511390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571c682-d963-4ff4-a330-65493ae8467c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:07.534246 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:07.549423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b06aa7-a87f-4ecf-adca-f21e5d8addfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:07.550361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06aa7-a87f-4ecf-adca-f21e5d8addfe None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:07.550645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06aa7-a87f-4ecf-adca-f21e5d8addfe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:07.550921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06aa7-a87f-4ecf-adca-f21e5d8addfe None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948/users/051466d6c63643fc83a5f8e1b0809fde/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:07.562564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b06aa7-a87f-4ecf-adca-f21e5d8addfe None admin] RBAC: Authorizing `identity:create_grant(project_id=affcd8ec317b49faaaa94fdb779a9948, user_id=051466d6c63643fc83a5f8e1b0809fde, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:07.564157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b06aa7-a87f-4ecf-adca-f21e5d8addfe None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:07.572305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea977e3-c4a2-41cf-abd3-85b6915a1329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:07.573398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea977e3-c4a2-41cf-abd3-85b6915a1329 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:07.573840 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea977e3-c4a2-41cf-abd3-85b6915a1329 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:07.574398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea977e3-c4a2-41cf-abd3-85b6915a1329 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:07.591534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 133/255] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:06:07 2022] PUT /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948/users/051466d6c63643fc83a5f8e1b0809fde/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:06:07.598388 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea977e3-c4a2-41cf-abd3-85b6915a1329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7ea977e3-c4a2-41cf-abd3-85b6915a1329 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:07.606321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:07.607323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:06:07.608198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:07.613152 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 123/256] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:07.614647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b06aa7-a87f-4ecf-adca-f21e5d8addfe None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:07.778664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b06aa7-a87f-4ecf-adca-f21e5d8addfe None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:07.827375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee48d0-221e-4861-8ecb-2fa08e87a95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:07.828224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee48d0-221e-4861-8ecb-2fa08e87a95a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:07.828513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee48d0-221e-4861-8ecb-2fa08e87a95a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:07.828795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee48d0-221e-4861-8ecb-2fa08e87a95a None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:07.829583 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee48d0-221e-4861-8ecb-2fa08e87a95a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:07.829921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee48d0-221e-4861-8ecb-2fa08e87a95a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:07.831360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee48d0-221e-4861-8ecb-2fa08e87a95a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:07.840715 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 134/257] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:07.847259 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea977e3-c4a2-41cf-abd3-85b6915a1329 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:07.994020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea977e3-c4a2-41cf-abd3-85b6915a1329 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:08.044626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552fe1c-4368-4030-bdd1-dfed2fbc1dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:08.046834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552fe1c-4368-4030-bdd1-dfed2fbc1dec None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:08.047322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552fe1c-4368-4030-bdd1-dfed2fbc1dec None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:08.047818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552fe1c-4368-4030-bdd1-dfed2fbc1dec None admin] PATH_INFO: `/v3/users/f3c4ee9213164a87a6278f251918bdd8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:08.049272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552fe1c-4368-4030-bdd1-dfed2fbc1dec None admin] RBAC: Authorizing `identity:get_user(user_id=f3c4ee9213164a87a6278f251918bdd8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:08.050983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552fe1c-4368-4030-bdd1-dfed2fbc1dec None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:08.052900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 124/258] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:07 2022] GET /identity/v3/users/f3c4ee9213164a87a6278f251918bdd8 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:08.058801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee48d0-221e-4861-8ecb-2fa08e87a95a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:08.126196 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:08.171267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aae70d-c857-4de2-9b7d-2051ee380514 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:08.171890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aae70d-c857-4de2-9b7d-2051ee380514 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:08.172374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aae70d-c857-4de2-9b7d-2051ee380514 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:08.174234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 125/259] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:08.180616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d44a68-4e58-4cb2-b8f4-81af5a4f3538 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:08.181340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d44a68-4e58-4cb2-b8f4-81af5a4f3538 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:08.181985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d44a68-4e58-4cb2-b8f4-81af5a4f3538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:08.213679 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b6d44a68-4e58-4cb2-b8f4-81af5a4f3538 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:08.219353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b6d44a68-4e58-4cb2-b8f4-81af5a4f3538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:08.256865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee48d0-221e-4861-8ecb-2fa08e87a95a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:08.316069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:08.321599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66cf03-3cf5-4f2c-b1c6-a07d30b1474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:08.322507 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66cf03-3cf5-4f2c-b1c6-a07d30b1474f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:08.322800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66cf03-3cf5-4f2c-b1c6-a07d30b1474f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:08.323078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66cf03-3cf5-4f2c-b1c6-a07d30b1474f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:08.400257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66cf03-3cf5-4f2c-b1c6-a07d30b1474f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:08.401745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66cf03-3cf5-4f2c-b1c6-a07d30b1474f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:08.403447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 126/260] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:06:08.446125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:08.453354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d44a68-4e58-4cb2-b8f4-81af5a4f3538 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:08.591058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d44a68-4e58-4cb2-b8f4-81af5a4f3538 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:08.591671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 135/261] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:08.598997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66cf03-3cf5-4f2c-b1c6-a07d30b1474f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:08.638243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252a3f-f295-423e-a2cb-47c7c5f13f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:08.640446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252a3f-f295-423e-a2cb-47c7c5f13f48 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:08.640921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252a3f-f295-423e-a2cb-47c7c5f13f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:08.641439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252a3f-f295-423e-a2cb-47c7c5f13f48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:08.656397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52252a3f-f295-423e-a2cb-47c7c5f13f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-52252a3f-f295-423e-a2cb-47c7c5f13f48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:08.666107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:08.667100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:08.667875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:08.670523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 127/262] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:06:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:08.677167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252a3f-f295-423e-a2cb-47c7c5f13f48 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:08.750076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:08.841621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66cf03-3cf5-4f2c-b1c6-a07d30b1474f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:08.890121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252a3f-f295-423e-a2cb-47c7c5f13f48 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:08.890595 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60a197-7734-4d3a-ba90-c36ad01375e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:08.891491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60a197-7734-4d3a-ba90-c36ad01375e3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:08.891775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60a197-7734-4d3a-ba90-c36ad01375e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:08.892053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60a197-7734-4d3a-ba90-c36ad01375e3 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:08.898667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60a197-7734-4d3a-ba90-c36ad01375e3 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:08.900170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60a197-7734-4d3a-ba90-c36ad01375e3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:08.902296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 136/263] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:08 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:08.910394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60a197-7734-4d3a-ba90-c36ad01375e3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:08.915162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:08.981262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f00560-c290-4da0-844c-33cc26da16c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:08.982005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f00560-c290-4da0-844c-33cc26da16c2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:08.982355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f00560-c290-4da0-844c-33cc26da16c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:08.982827 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f00560-c290-4da0-844c-33cc26da16c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:08.983777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f00560-c290-4da0-844c-33cc26da16c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:08.984243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f00560-c290-4da0-844c-33cc26da16c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:08.985703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f00560-c290-4da0-844c-33cc26da16c2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:09.011777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 128/264] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:09.017549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f00560-c290-4da0-844c-33cc26da16c2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:09.168978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60a197-7734-4d3a-ba90-c36ad01375e3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:09.169512 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:09.174898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f00560-c290-4da0-844c-33cc26da16c2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:09.249572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33250218-db5f-40f8-b2b8-a20790d62941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:09.250505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33250218-db5f-40f8-b2b8-a20790d62941 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:09.250793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33250218-db5f-40f8-b2b8-a20790d62941 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:09.251073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33250218-db5f-40f8-b2b8-a20790d62941 None admin] PATH_INFO: `/v3/users/f3c4ee9213164a87a6278f251918bdd8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:09.252758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33250218-db5f-40f8-b2b8-a20790d62941 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c4ee9213164a87a6278f251918bdd8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:09.254253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33250218-db5f-40f8-b2b8-a20790d62941 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:09.255927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 129/265] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:09 2022] GET /identity/v3/users/f3c4ee9213164a87a6278f251918bdd8 => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:09.262767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33250218-db5f-40f8-b2b8-a20790d62941 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:09.263743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2c145-87e2-4e0f-b3cf-0dfedf1f8405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:09.265258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2c145-87e2-4e0f-b3cf-0dfedf1f8405 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:09.265258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2c145-87e2-4e0f-b3cf-0dfedf1f8405 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:09.265635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2c145-87e2-4e0f-b3cf-0dfedf1f8405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:09.266414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2c145-87e2-4e0f-b3cf-0dfedf1f8405 None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=affcd8ec317b49faaaa94fdb779a9948, user.id=f3c4ee9213164a87a6278f251918bdd8) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:09.266720 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2c145-87e2-4e0f-b3cf-0dfedf1f8405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:09.268201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2c145-87e2-4e0f-b3cf-0dfedf1f8405 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:09.288278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 137/266] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:08 2022] GET /identity/v3/role_assignments?user.id=f3c4ee9213164a87a6278f251918bdd8&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 608 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:09.427826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33250218-db5f-40f8-b2b8-a20790d62941 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:09.477408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204026d9-eadd-4638-97c6-f420f64bcd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:09.478120 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204026d9-eadd-4638-97c6-f420f64bcd6d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:09.478409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204026d9-eadd-4638-97c6-f420f64bcd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:09.478692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204026d9-eadd-4638-97c6-f420f64bcd6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:09.536835 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:09.577743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204026d9-eadd-4638-97c6-f420f64bcd6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:09.579293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204026d9-eadd-4638-97c6-f420f64bcd6d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:09.696460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 130/267] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:09.701729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2c145-87e2-4e0f-b3cf-0dfedf1f8405 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:09.825818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:09.870144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2c145-87e2-4e0f-b3cf-0dfedf1f8405 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:09.917248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5680d1-d363-48f2-b1ac-1c7fa2dfcf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:09.918170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5680d1-d363-48f2-b1ac-1c7fa2dfcf3c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:09.918452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5680d1-d363-48f2-b1ac-1c7fa2dfcf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:09.918734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5680d1-d363-48f2-b1ac-1c7fa2dfcf3c None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:09.919962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5680d1-d363-48f2-b1ac-1c7fa2dfcf3c None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:09.921468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5680d1-d363-48f2-b1ac-1c7fa2dfcf3c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:09.923543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 138/268] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:09 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:09.929401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204026d9-eadd-4638-97c6-f420f64bcd6d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:10.021606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:10.074714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204026d9-eadd-4638-97c6-f420f64bcd6d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:10.128875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8bdd4-793c-4be7-a32c-dec805984daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:10.129553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8bdd4-793c-4be7-a32c-dec805984daf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:10.129862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8bdd4-793c-4be7-a32c-dec805984daf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:10.130141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8bdd4-793c-4be7-a32c-dec805984daf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:10.130974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8bdd4-793c-4be7-a32c-dec805984daf None admin] RBAC: Adding query filter params (role.id=e7180b44fc2945568dfb83dfaf8b5c79, scope.project.id=affcd8ec317b49faaaa94fdb779a9948, user.id=f3c4ee9213164a87a6278f251918bdd8) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:10.131316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8bdd4-793c-4be7-a32c-dec805984daf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:10.132758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8bdd4-793c-4be7-a32c-dec805984daf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:10.144093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 131/269] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:09 2022] GET /identity/v3/role_assignments?user.id=f3c4ee9213164a87a6278f251918bdd8&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=e7180b44fc2945568dfb83dfaf8b5c79 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:10.322112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4d4c5-5081-467f-9cf7-2205b894170f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:10.322474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4d4c5-5081-467f-9cf7-2205b894170f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:10.322700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4d4c5-5081-467f-9cf7-2205b894170f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:10.327106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 139/270] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:10.331159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfa70a-3932-4ef1-a3e5-b36973aa4d47 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:10.331696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfa70a-3932-4ef1-a3e5-b36973aa4d47 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:10.331696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfa70a-3932-4ef1-a3e5-b36973aa4d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:10.354870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cdbfa70a-3932-4ef1-a3e5-b36973aa4d47 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:10.356091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cdbfa70a-3932-4ef1-a3e5-b36973aa4d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:10.437732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 132/271] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:10.479438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4d4c5-5081-467f-9cf7-2205b894170f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:10.504331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da64b9-59d0-4d17-ae75-4d34c92f418b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:10.504631 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da64b9-59d0-4d17-ae75-4d34c92f418b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:10.504886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da64b9-59d0-4d17-ae75-4d34c92f418b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:10.506597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 133/272] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:10.512422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e940-67b6-4f3d-956b-f36d083dc3ed None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:10.512422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e940-67b6-4f3d-956b-f36d083dc3ed None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:10.512625 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e940-67b6-4f3d-956b-f36d083dc3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:10.533073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d048e940-67b6-4f3d-956b-f36d083dc3ed None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:10.534570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d048e940-67b6-4f3d-956b-f36d083dc3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:10.611779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4d4c5-5081-467f-9cf7-2205b894170f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:10.616630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:10.632655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 134/273] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:06:10.634548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0370ef-965f-4dda-a1b9-b94134844dde None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:10.635075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0370ef-965f-4dda-a1b9-b94134844dde None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:10.635363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0370ef-965f-4dda-a1b9-b94134844dde None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:10.636673 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 135/274] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:10.642937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aa933-5947-43b4-b7a4-9180e50f328b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:10.643076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aa933-5947-43b4-b7a4-9180e50f328b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:10.643399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aa933-5947-43b4-b7a4-9180e50f328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:10.663439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-aa0aa933-5947-43b4-b7a4-9180e50f328b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:10.664247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221abb16-1a70-42a2-953f-092efaa1245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:10.664638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aa0aa933-5947-43b4-b7a4-9180e50f328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:10.665598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221abb16-1a70-42a2-953f-092efaa1245a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:10.666211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221abb16-1a70-42a2-953f-092efaa1245a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:10.666702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221abb16-1a70-42a2-953f-092efaa1245a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:10.669270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221abb16-1a70-42a2-953f-092efaa1245a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:10.671180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221abb16-1a70-42a2-953f-092efaa1245a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:10.674087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 140/275] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:10.675194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221abb16-1a70-42a2-953f-092efaa1245a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:10.679831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:10.740136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 136/276] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:10.740767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0aa933-5947-43b4-b7a4-9180e50f328b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:10.801838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221abb16-1a70-42a2-953f-092efaa1245a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:10.845752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54fa3-acf0-4690-a728-8d439822f8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:10.846915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54fa3-acf0-4690-a728-8d439822f8ae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:10.847391 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54fa3-acf0-4690-a728-8d439822f8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:10.847668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54fa3-acf0-4690-a728-8d439822f8ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:10.857270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0aa933-5947-43b4-b7a4-9180e50f328b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:10.861684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f54fa3-acf0-4690-a728-8d439822f8ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-45f54fa3-acf0-4690-a728-8d439822f8ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:10.869625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:10.870689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 141/277] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:10.875063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54fa3-acf0-4690-a728-8d439822f8ae None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:10.886503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:10.903130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c60ea-b9b0-4119-8118-04165aa9654e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:10.903833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c60ea-b9b0-4119-8118-04165aa9654e None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:10.904095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c60ea-b9b0-4119-8118-04165aa9654e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:10.904454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c60ea-b9b0-4119-8118-04165aa9654e None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:10.905960 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c60ea-b9b0-4119-8118-04165aa9654e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:10.907462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c60ea-b9b0-4119-8118-04165aa9654e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:10.929751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 137/278] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:06:10 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:06:10.930598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c60ea-b9b0-4119-8118-04165aa9654e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:10.953259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:11.005455 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54fa3-acf0-4690-a728-8d439822f8ae None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:11.049441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178203e-0c7c-46ca-bd2c-de75755b13c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:11.050376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178203e-0c7c-46ca-bd2c-de75755b13c3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.050734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178203e-0c7c-46ca-bd2c-de75755b13c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.051109 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178203e-0c7c-46ca-bd2c-de75755b13c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.064360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178203e-0c7c-46ca-bd2c-de75755b13c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e178203e-0c7c-46ca-bd2c-de75755b13c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:11.072567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:11.073543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:11.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 142/279] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:11.078654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c60ea-b9b0-4119-8118-04165aa9654e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:11.082823 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178203e-0c7c-46ca-bd2c-de75755b13c3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:11.133629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48fb54-e9bd-45fa-9945-eb6caff59dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:11.134147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48fb54-e9bd-45fa-9945-eb6caff59dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.134562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48fb54-e9bd-45fa-9945-eb6caff59dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.134880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48fb54-e9bd-45fa-9945-eb6caff59dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.135751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48fb54-e9bd-45fa-9945-eb6caff59dc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:11.136068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48fb54-e9bd-45fa-9945-eb6caff59dc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:11.137776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48fb54-e9bd-45fa-9945-eb6caff59dc1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:11.153578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 138/280] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:11.154723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48fb54-e9bd-45fa-9945-eb6caff59dc1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:11.217509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:11.241279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178203e-0c7c-46ca-bd2c-de75755b13c3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:11.272929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:11.291352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ee42c-2d26-4e3f-8c61-552e067dca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:11.292599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee42c-2d26-4e3f-8c61-552e067dca48 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.293270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee42c-2d26-4e3f-8c61-552e067dca48 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.293680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee42c-2d26-4e3f-8c61-552e067dca48 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.295648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ee42c-2d26-4e3f-8c61-552e067dca48 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:11.296758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ee42c-2d26-4e3f-8c61-552e067dca48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:11.298424 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ee42c-2d26-4e3f-8c61-552e067dca48 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:11.301607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48fb54-e9bd-45fa-9945-eb6caff59dc1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:11.311552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 143/281] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:11.317403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ee42c-2d26-4e3f-8c61-552e067dca48 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:11.350425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403f5c6-7e94-4b90-a8d0-8ea502028196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:11.351329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403f5c6-7e94-4b90-a8d0-8ea502028196 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.351784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403f5c6-7e94-4b90-a8d0-8ea502028196 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.351990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403f5c6-7e94-4b90-a8d0-8ea502028196 None admin] PATH_INFO: `/v3/users/43709c4fbdc944198fd4e2005d1b0e6c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.369921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403f5c6-7e94-4b90-a8d0-8ea502028196 None admin] RBAC: Authorizing `identity:get_user(user_id=43709c4fbdc944198fd4e2005d1b0e6c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:11.371383 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403f5c6-7e94-4b90-a8d0-8ea502028196 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:11.374047 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 139/282] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:11 2022] GET /identity/v3/users/43709c4fbdc944198fd4e2005d1b0e6c => generated 293 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:11.375597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fef5b6-5a99-4e05-aca9-bc4343523d96 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.375883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fef5b6-5a99-4e05-aca9-bc4343523d96 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.376157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fef5b6-5a99-4e05-aca9-bc4343523d96 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.377527 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 140/283] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:11.386520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9900bf1-0d90-4a14-afbd-f2e456b8fb92 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.386772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9900bf1-0d90-4a14-afbd-f2e456b8fb92 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.387049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9900bf1-0d90-4a14-afbd-f2e456b8fb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.406968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e9900bf1-0d90-4a14-afbd-f2e456b8fb92 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:11.408145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e9900bf1-0d90-4a14-afbd-f2e456b8fb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:11.458142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ee42c-2d26-4e3f-8c61-552e067dca48 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:11.487547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 141/284] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:11.488660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9900bf1-0d90-4a14-afbd-f2e456b8fb92 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:11.509163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bd7c6-64be-4bac-bb1a-78d8d8e31538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:11.511026 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bd7c6-64be-4bac-bb1a-78d8d8e31538 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.512128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bd7c6-64be-4bac-bb1a-78d8d8e31538 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.512619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bd7c6-64be-4bac-bb1a-78d8d8e31538 None admin] PATH_INFO: `/v3/users/5ebdc65cf1db44c4b88fc4125e13e39a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.517322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bd7c6-64be-4bac-bb1a-78d8d8e31538 None admin] RBAC: Authorizing `identity:get_user(user_id=5ebdc65cf1db44c4b88fc4125e13e39a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:11.518973 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bd7c6-64be-4bac-bb1a-78d8d8e31538 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:11.521905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 144/285] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:11 2022] GET /identity/v3/users/5ebdc65cf1db44c4b88fc4125e13e39a => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:11.523023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bd7c6-64be-4bac-bb1a-78d8d8e31538 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:11.529037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:11.591990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:11.632964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9900bf1-0d90-4a14-afbd-f2e456b8fb92 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:11.651022 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bd7c6-64be-4bac-bb1a-78d8d8e31538 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:11.672256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d116-a5e8-4641-b0c7-0040f4766026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:11.672926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d116-a5e8-4641-b0c7-0040f4766026 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.673206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d116-a5e8-4641-b0c7-0040f4766026 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.673478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d116-a5e8-4641-b0c7-0040f4766026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.674437 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378d116-a5e8-4641-b0c7-0040f4766026 None admin] RBAC: Adding query filter params (role.id=7f6dd280cf8941dcb8f79ecf88dbda78, user.id=43709c4fbdc944198fd4e2005d1b0e6c, scope.system=all) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:11.674747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378d116-a5e8-4641-b0c7-0040f4766026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:11.676305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378d116-a5e8-4641-b0c7-0040f4766026 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:11.682307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 142/286] 10.222.0.42 () {56 vars in 1308 bytes} [Wed May 25 16:06:11 2022] GET /identity/v3/role_assignments?user.id=43709c4fbdc944198fd4e2005d1b0e6c&scope.system=all&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:11.683275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d116-a5e8-4641-b0c7-0040f4766026 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:11.701597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdc542-67dd-4b79-92a4-db3e82246c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:11.701597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdc542-67dd-4b79-92a4-db3e82246c9d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.701597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdc542-67dd-4b79-92a4-db3e82246c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.701597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdc542-67dd-4b79-92a4-db3e82246c9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.727145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdc542-67dd-4b79-92a4-db3e82246c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1efdc542-67dd-4b79-92a4-db3e82246c9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:11.742067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:11.743195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:11.744092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 145/287] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:11.750316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdc542-67dd-4b79-92a4-db3e82246c9d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:11.832783 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:11.872335 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d116-a5e8-4641-b0c7-0040f4766026 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:11.888507 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdc542-67dd-4b79-92a4-db3e82246c9d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:11.913274 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:11.941737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcac6b-a321-4c22-aab4-96fd043715cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:11.942095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5e36c-4077-4a3b-9b97-9cc10b522f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:11.942986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5e36c-4077-4a3b-9b97-9cc10b522f1a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.943114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcac6b-a321-4c22-aab4-96fd043715cc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.943272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5e36c-4077-4a3b-9b97-9cc10b522f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.943540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5e36c-4077-4a3b-9b97-9cc10b522f1a None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.943920 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcac6b-a321-4c22-aab4-96fd043715cc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.944381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcac6b-a321-4c22-aab4-96fd043715cc None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.944745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5e36c-4077-4a3b-9b97-9cc10b522f1a None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:11.945529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcac6b-a321-4c22-aab4-96fd043715cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:11.946077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcac6b-a321-4c22-aab4-96fd043715cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:11.946335 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5e36c-4077-4a3b-9b97-9cc10b522f1a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:11.947832 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcac6b-a321-4c22-aab4-96fd043715cc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:11.948389 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 143/288] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:11 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:11.954288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5e36c-4077-4a3b-9b97-9cc10b522f1a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:11.960190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 146/289] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:11.965582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcac6b-a321-4c22-aab4-96fd043715cc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:12.072263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5e36c-4077-4a3b-9b97-9cc10b522f1a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:12.100358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcac6b-a321-4c22-aab4-96fd043715cc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:12.111469 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b2c82-f9c2-4cc8-b5a2-341121f68563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:12.112781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2c82-f9c2-4cc8-b5a2-341121f68563 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:12.113309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2c82-f9c2-4cc8-b5a2-341121f68563 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:12.113807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2c82-f9c2-4cc8-b5a2-341121f68563 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948/users/5ebdc65cf1db44c4b88fc4125e13e39a/roles/a200b6cb596344e385e5f234488c6018` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:12.116205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b2c82-f9c2-4cc8-b5a2-341121f68563 None admin] RBAC: Authorizing `identity:create_grant(project_id=affcd8ec317b49faaaa94fdb779a9948, user_id=5ebdc65cf1db44c4b88fc4125e13e39a, role_id=a200b6cb596344e385e5f234488c6018)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:12.117905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b2c82-f9c2-4cc8-b5a2-341121f68563 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:12.134470 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 144/290] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:06:11 2022] PUT /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948/users/5ebdc65cf1db44c4b88fc4125e13e39a/roles/a200b6cb596344e385e5f234488c6018 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:06:12.143786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477610cb-3a3f-4024-9abe-6e2e6382209e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:12.144814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477610cb-3a3f-4024-9abe-6e2e6382209e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:12.145103 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477610cb-3a3f-4024-9abe-6e2e6382209e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:12.145381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477610cb-3a3f-4024-9abe-6e2e6382209e None admin] PATH_INFO: `/v3/users/051466d6c63643fc83a5f8e1b0809fde` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:12.146805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477610cb-3a3f-4024-9abe-6e2e6382209e None admin] RBAC: Authorizing `identity:get_user(user_id=051466d6c63643fc83a5f8e1b0809fde)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:12.148363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477610cb-3a3f-4024-9abe-6e2e6382209e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:12.150483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 147/291] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:11 2022] GET /identity/v3/users/051466d6c63643fc83a5f8e1b0809fde => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:12.156204 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b2c82-f9c2-4cc8-b5a2-341121f68563 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:12.195496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:12.309010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b2c82-f9c2-4cc8-b5a2-341121f68563 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:12.364060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4cfe0-b5ed-4377-950d-3cdf366c747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:12.365604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4cfe0-b5ed-4377-950d-3cdf366c747b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:12.366255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4cfe0-b5ed-4377-950d-3cdf366c747b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:12.366725 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4cfe0-b5ed-4377-950d-3cdf366c747b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:12.484389 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4cfe0-b5ed-4377-950d-3cdf366c747b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:12.485885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4cfe0-b5ed-4377-950d-3cdf366c747b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:12.516334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:12.618316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 145/292] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 16:06:12.624285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477610cb-3a3f-4024-9abe-6e2e6382209e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:12.679199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:12.742945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477610cb-3a3f-4024-9abe-6e2e6382209e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:12.794605 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1826c9e-8a83-4d72-ad66-52817592c314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:12.796230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1826c9e-8a83-4d72-ad66-52817592c314 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:12.796707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1826c9e-8a83-4d72-ad66-52817592c314 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:12.796988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1826c9e-8a83-4d72-ad66-52817592c314 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:12.798232 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1826c9e-8a83-4d72-ad66-52817592c314 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:12.799738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1826c9e-8a83-4d72-ad66-52817592c314 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:12.801766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 148/293] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:12 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:12.807338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4cfe0-b5ed-4377-950d-3cdf366c747b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:12.932500 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4cfe0-b5ed-4377-950d-3cdf366c747b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:12.966077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:12.984025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba15b41-b55e-4f6e-bce1-289d68f4b20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:12.984694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba15b41-b55e-4f6e-bce1-289d68f4b20f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:12.984974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba15b41-b55e-4f6e-bce1-289d68f4b20f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:12.985245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba15b41-b55e-4f6e-bce1-289d68f4b20f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:12.986119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba15b41-b55e-4f6e-bce1-289d68f4b20f None admin] RBAC: Adding query filter params (role.id=7f6dd280cf8941dcb8f79ecf88dbda78, scope.project.id=affcd8ec317b49faaaa94fdb779a9948, user.id=051466d6c63643fc83a5f8e1b0809fde) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:12.986427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba15b41-b55e-4f6e-bce1-289d68f4b20f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:12.987859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba15b41-b55e-4f6e-bce1-289d68f4b20f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:12.994521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 146/294] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:12 2022] GET /identity/v3/role_assignments?user.id=051466d6c63643fc83a5f8e1b0809fde&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:13.769705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d2770-8777-4921-bfe0-497f1a07e721 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:13.770211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d2770-8777-4921-bfe0-497f1a07e721 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:13.770211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d2770-8777-4921-bfe0-497f1a07e721 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:13.771845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 149/295] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:13.777476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63210e20-b416-4a9a-8ff7-6c635639ce2a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:13.777983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63210e20-b416-4a9a-8ff7-6c635639ce2a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:13.778058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63210e20-b416-4a9a-8ff7-6c635639ce2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:13.797570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-63210e20-b416-4a9a-8ff7-6c635639ce2a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:13.799175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-63210e20-b416-4a9a-8ff7-6c635639ce2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:13.870836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 147/296] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:13.895066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d2770-8777-4921-bfe0-497f1a07e721 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:13.896844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa32336-153c-48ed-a54c-bc9f7a27daa0 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:13.897343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa32336-153c-48ed-a54c-bc9f7a27daa0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:13.897976 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa32336-153c-48ed-a54c-bc9f7a27daa0 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:13.899955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 148/297] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:13.905601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e480f-d149-4c0c-9d6a-1ba567b767c3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:13.906155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e480f-d149-4c0c-9d6a-1ba567b767c3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:13.906631 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e480f-d149-4c0c-9d6a-1ba567b767c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:13.926124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f52e480f-d149-4c0c-9d6a-1ba567b767c3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:13.927752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f52e480f-d149-4c0c-9d6a-1ba567b767c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:14.007548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d2770-8777-4921-bfe0-497f1a07e721 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:14.007937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 149/298] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:14.017075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:14.030660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e480f-d149-4c0c-9d6a-1ba567b767c3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:14.043914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49bff8-df9f-40d9-a56f-76ad71692fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:14.045583 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bff8-df9f-40d9-a56f-76ad71692fed None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.045583 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bff8-df9f-40d9-a56f-76ad71692fed None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.045904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bff8-df9f-40d9-a56f-76ad71692fed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.058180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49bff8-df9f-40d9-a56f-76ad71692fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ff49bff8-df9f-40d9-a56f-76ad71692fed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:14.068294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:14.069500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:14.071642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 150/299] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:14.077383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49bff8-df9f-40d9-a56f-76ad71692fed None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:14.098154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:14.156531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e480f-d149-4c0c-9d6a-1ba567b767c3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:14.187388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49bff8-df9f-40d9-a56f-76ad71692fed None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:14.195505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67248f25-58c6-4b07-acf8-13aa6cc53ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:14.196620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67248f25-58c6-4b07-acf8-13aa6cc53ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.197232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67248f25-58c6-4b07-acf8-13aa6cc53ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.197704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67248f25-58c6-4b07-acf8-13aa6cc53ca1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.199126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67248f25-58c6-4b07-acf8-13aa6cc53ca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:14.200936 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67248f25-58c6-4b07-acf8-13aa6cc53ca1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:14.208303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 150/300] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:14.209818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67248f25-58c6-4b07-acf8-13aa6cc53ca1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:14.226474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28903fe-10cd-4eb3-8fee-2ed7fbe277ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:14.227380 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28903fe-10cd-4eb3-8fee-2ed7fbe277ba None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.227491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28903fe-10cd-4eb3-8fee-2ed7fbe277ba None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.227696 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28903fe-10cd-4eb3-8fee-2ed7fbe277ba None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.228453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28903fe-10cd-4eb3-8fee-2ed7fbe277ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:14.228766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28903fe-10cd-4eb3-8fee-2ed7fbe277ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:14.230271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28903fe-10cd-4eb3-8fee-2ed7fbe277ba None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:14.239867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 151/301] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:14.241294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f45859-a2e8-4c2a-8449-a09b1e6f01c7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.241596 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f45859-a2e8-4c2a-8449-a09b1e6f01c7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.241903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f45859-a2e8-4c2a-8449-a09b1e6f01c7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.243155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 152/302] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:14.246156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f45859-a2e8-4c2a-8449-a09b1e6f01c7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:14.278972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:14.321593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67248f25-58c6-4b07-acf8-13aa6cc53ca1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:14.332922 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:14.359316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e6845-e6be-44c0-990a-d49459092f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:14.359891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f45859-a2e8-4c2a-8449-a09b1e6f01c7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:14.360079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e6845-e6be-44c0-990a-d49459092f7c None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.360325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e6845-e6be-44c0-990a-d49459092f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.360605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e6845-e6be-44c0-990a-d49459092f7c None admin] PATH_INFO: `/v3/groups` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.361302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e6845-e6be-44c0-990a-d49459092f7c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:14.362779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e6845-e6be-44c0-990a-d49459092f7c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:14.378598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 151/303] 10.222.0.42 () {60 vars in 1134 bytes} [Wed May 25 16:06:14 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:06:14.378598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4a5f-2c08-4a4d-8029-ccc90ed4af5d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.378598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4a5f-2c08-4a4d-8029-ccc90ed4af5d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.378598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4a5f-2c08-4a4d-8029-ccc90ed4af5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.406006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2d53d-418a-4130-8f3a-9d53a55945f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:14.407185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2d53d-418a-4130-8f3a-9d53a55945f5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.407524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2d53d-418a-4130-8f3a-9d53a55945f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.407799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2d53d-418a-4130-8f3a-9d53a55945f5 None admin] PATH_INFO: `/v3/users/6e11df1b2fcb45cbb4a927e2dccf2b0d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.413547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b93e4a5f-2c08-4a4d-8029-ccc90ed4af5d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:14.413547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b93e4a5f-2c08-4a4d-8029-ccc90ed4af5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:14.429399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2d53d-418a-4130-8f3a-9d53a55945f5 None admin] RBAC: Authorizing `identity:get_user(user_id=6e11df1b2fcb45cbb4a927e2dccf2b0d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:14.438511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2d53d-418a-4130-8f3a-9d53a55945f5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:14.439073 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 153/304] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:14 2022] GET /identity/v3/users/6e11df1b2fcb45cbb4a927e2dccf2b0d => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:14.440058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2d53d-418a-4130-8f3a-9d53a55945f5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:14.518026 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 152/305] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:14.541092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e4a5f-2c08-4a4d-8029-ccc90ed4af5d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:14.553293 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:14.565843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2d53d-418a-4130-8f3a-9d53a55945f5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:14.608029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50925a6a-5b58-4b5c-ba83-af9c9748f4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:14.608911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50925a6a-5b58-4b5c-ba83-af9c9748f4db None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.609349 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50925a6a-5b58-4b5c-ba83-af9c9748f4db None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.609816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50925a6a-5b58-4b5c-ba83-af9c9748f4db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.611281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50925a6a-5b58-4b5c-ba83-af9c9748f4db None admin] RBAC: Adding query filter params (role.id=a200b6cb596344e385e5f234488c6018, user.id=6e11df1b2fcb45cbb4a927e2dccf2b0d, scope.system=all) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:14.611749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50925a6a-5b58-4b5c-ba83-af9c9748f4db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:14.614104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50925a6a-5b58-4b5c-ba83-af9c9748f4db None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:14.621569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 154/306] 10.222.0.42 () {56 vars in 1308 bytes} [Wed May 25 16:06:14 2022] GET /identity/v3/role_assignments?user.id=6e11df1b2fcb45cbb4a927e2dccf2b0d&scope.system=all&role.id=a200b6cb596344e385e5f234488c6018 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:14.678382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e4a5f-2c08-4a4d-8029-ccc90ed4af5d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:14.680301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:14.722919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4700f-f0f1-49ad-8ebc-d4691a755d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:14.723755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4700f-f0f1-49ad-8ebc-d4691a755d48 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.724034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4700f-f0f1-49ad-8ebc-d4691a755d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.724307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4700f-f0f1-49ad-8ebc-d4691a755d48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.738410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4700f-f0f1-49ad-8ebc-d4691a755d48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-63d4700f-f0f1-49ad-8ebc-d4691a755d48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:14.746450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:14.747497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:14.748812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 153/307] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:14.754318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50925a6a-5b58-4b5c-ba83-af9c9748f4db None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:14.874535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50925a6a-5b58-4b5c-ba83-af9c9748f4db None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:14.912632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b91368-e9b1-4462-a177-3565bf5499f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:14.913686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91368-e9b1-4462-a177-3565bf5499f5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.914203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91368-e9b1-4462-a177-3565bf5499f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.914684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91368-e9b1-4462-a177-3565bf5499f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.915679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b91368-e9b1-4462-a177-3565bf5499f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:14.916187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b91368-e9b1-4462-a177-3565bf5499f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:14.917857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b91368-e9b1-4462-a177-3565bf5499f5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:14.928768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 155/308] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:14.934016 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4700f-f0f1-49ad-8ebc-d4691a755d48 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:14.941931 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462028e9-0a95-410d-966f-818452832e40 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.942226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462028e9-0a95-410d-966f-818452832e40 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.942511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462028e9-0a95-410d-966f-818452832e40 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.944143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 156/309] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:14.949302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0915c2-f9ea-412d-8c41-5bdf02f69db6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:14.949787 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0915c2-f9ea-412d-8c41-5bdf02f69db6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:14.949912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0915c2-f9ea-412d-8c41-5bdf02f69db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:14.976313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7f0915c2-f9ea-412d-8c41-5bdf02f69db6 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:14.978463 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7f0915c2-f9ea-412d-8c41-5bdf02f69db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:14.993985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:15.056939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4700f-f0f1-49ad-8ebc-d4691a755d48 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:15.063524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 157/310] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:06:15.086672 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0915c2-f9ea-412d-8c41-5bdf02f69db6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:15.096254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47735f-7de0-4ba8-b6f6-72b6deee94d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:15.097511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47735f-7de0-4ba8-b6f6-72b6deee94d1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:15.098067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47735f-7de0-4ba8-b6f6-72b6deee94d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:15.098702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47735f-7de0-4ba8-b6f6-72b6deee94d1 None admin] PATH_INFO: `/v3/users/43709c4fbdc944198fd4e2005d1b0e6c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:15.100245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47735f-7de0-4ba8-b6f6-72b6deee94d1 None admin] RBAC: Authorizing `identity:get_user(user_id=43709c4fbdc944198fd4e2005d1b0e6c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:15.101989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47735f-7de0-4ba8-b6f6-72b6deee94d1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:15.104463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 154/311] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:14 2022] GET /identity/v3/users/43709c4fbdc944198fd4e2005d1b0e6c => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:15.110945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47735f-7de0-4ba8-b6f6-72b6deee94d1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:15.152714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:15.211353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0915c2-f9ea-412d-8c41-5bdf02f69db6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:15.238644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47735f-7de0-4ba8-b6f6-72b6deee94d1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:15.266706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52834c61-b9cd-4e04-a685-18c079445bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:15.267506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52834c61-b9cd-4e04-a685-18c079445bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:15.267779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52834c61-b9cd-4e04-a685-18c079445bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:15.268067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52834c61-b9cd-4e04-a685-18c079445bd0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:15.280979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52834c61-b9cd-4e04-a685-18c079445bd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:15.283677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fa5bf-2515-436c-a747-aea22d27fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:15.285044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fa5bf-2515-436c-a747-aea22d27fd03 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:15.285549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fa5bf-2515-436c-a747-aea22d27fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:15.286059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fa5bf-2515-436c-a747-aea22d27fd03 None admin] PATH_INFO: `/v3/system/users/43709c4fbdc944198fd4e2005d1b0e6c/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:15.287824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fa5bf-2515-436c-a747-aea22d27fd03 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=43709c4fbdc944198fd4e2005d1b0e6c, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-52834c61-b9cd-4e04-a685-18c079445bd0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:15.288968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:15.289894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:15.290702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 158/312] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:15.297116 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fa5bf-2515-436c-a747-aea22d27fd03 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:15.297683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52834c61-b9cd-4e04-a685-18c079445bd0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:15.308604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 155/313] 10.222.0.42 () {58 vars in 1334 bytes} [Wed May 25 16:06:15 2022] PUT /identity/v3/system/users/43709c4fbdc944198fd4e2005d1b0e6c/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:06:15.310828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:15.433422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52834c61-b9cd-4e04-a685-18c079445bd0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:15.473185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3252945-3688-4003-92b9-d058411b5e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:15.473891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3252945-3688-4003-92b9-d058411b5e12 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:15.474176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3252945-3688-4003-92b9-d058411b5e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:15.474448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3252945-3688-4003-92b9-d058411b5e12 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:15.475210 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3252945-3688-4003-92b9-d058411b5e12 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:15.475513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3252945-3688-4003-92b9-d058411b5e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:15.476958 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3252945-3688-4003-92b9-d058411b5e12 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:15.487479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 159/314] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:15.493457 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fa5bf-2515-436c-a747-aea22d27fd03 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:15.599935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:15.610912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fa5bf-2515-436c-a747-aea22d27fd03 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:15.648694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe401d-208e-42bf-bf99-5072e97e3d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:15.649568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe401d-208e-42bf-bf99-5072e97e3d00 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:15.649874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe401d-208e-42bf-bf99-5072e97e3d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:15.650144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe401d-208e-42bf-bf99-5072e97e3d00 None admin] PATH_INFO: `/v3/users/5ebdc65cf1db44c4b88fc4125e13e39a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:15.651333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe401d-208e-42bf-bf99-5072e97e3d00 None admin] RBAC: Authorizing `identity:get_user(user_id=5ebdc65cf1db44c4b88fc4125e13e39a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:15.652806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe401d-208e-42bf-bf99-5072e97e3d00 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:15.655995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 156/315] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:15 2022] GET /identity/v3/users/5ebdc65cf1db44c4b88fc4125e13e39a => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:15.662918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3252945-3688-4003-92b9-d058411b5e12 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:15.737228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:15.774820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3252945-3688-4003-92b9-d058411b5e12 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:15.812482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e72a9c-b818-4633-b171-dddf0b8d69de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:15.813155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e72a9c-b818-4633-b171-dddf0b8d69de None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:15.813437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e72a9c-b818-4633-b171-dddf0b8d69de None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:15.813732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e72a9c-b818-4633-b171-dddf0b8d69de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:15.887578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e72a9c-b818-4633-b171-dddf0b8d69de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:15.890055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e72a9c-b818-4633-b171-dddf0b8d69de None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:15.968482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:16.010733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 160/316] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:16.016832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe401d-208e-42bf-bf99-5072e97e3d00 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:16.129441 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe401d-208e-42bf-bf99-5072e97e3d00 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:16.166258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9fbf5-970b-4b00-a6c4-80b6c2cd22ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:16.167135 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9fbf5-970b-4b00-a6c4-80b6c2cd22ea None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:16.167418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9fbf5-970b-4b00-a6c4-80b6c2cd22ea None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:16.167694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9fbf5-970b-4b00-a6c4-80b6c2cd22ea None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:16.168907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9fbf5-970b-4b00-a6c4-80b6c2cd22ea None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:16.170424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9fbf5-970b-4b00-a6c4-80b6c2cd22ea None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:16.172145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 157/317] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:16 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:16.178016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e72a9c-b818-4633-b171-dddf0b8d69de None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:16.285609 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e72a9c-b818-4633-b171-dddf0b8d69de None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:16.328371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ebc52-c8ec-4c70-be01-ca57370ea994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:16.329047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ebc52-c8ec-4c70-be01-ca57370ea994 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:16.329336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ebc52-c8ec-4c70-be01-ca57370ea994 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:16.329614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ebc52-c8ec-4c70-be01-ca57370ea994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:16.330500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ebc52-c8ec-4c70-be01-ca57370ea994 None admin] RBAC: Adding query filter params (role.id=a200b6cb596344e385e5f234488c6018, scope.project.id=affcd8ec317b49faaaa94fdb779a9948, user.id=5ebdc65cf1db44c4b88fc4125e13e39a) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:16.330816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ebc52-c8ec-4c70-be01-ca57370ea994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:16.333443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ebc52-c8ec-4c70-be01-ca57370ea994 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:16.343465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 161/318] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:16 2022] GET /identity/v3/role_assignments?user.id=5ebdc65cf1db44c4b88fc4125e13e39a&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=a200b6cb596344e385e5f234488c6018 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:16.876820 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673511f-e277-4a6f-9403-5e0e02581590 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:16.877123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673511f-e277-4a6f-9403-5e0e02581590 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:16.877531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673511f-e277-4a6f-9403-5e0e02581590 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:16.879078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 158/319] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:16.884116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c60f0-564a-4439-a6dc-436e01bcb02f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:16.884485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c60f0-564a-4439-a6dc-436e01bcb02f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:16.884840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c60f0-564a-4439-a6dc-436e01bcb02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:16.902312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-242c60f0-564a-4439-a6dc-436e01bcb02f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:16.903496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-242c60f0-564a-4439-a6dc-436e01bcb02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:16.932265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:16.965392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2099ccc-6337-45a1-9a26-68d44929cf68 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:16.965763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2099ccc-6337-45a1-9a26-68d44929cf68 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:16.965990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2099ccc-6337-45a1-9a26-68d44929cf68 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:16.968646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 159/320] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:16.974013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce6f87-815f-4e78-bfb8-885f23cb4d70 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:16.974304 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce6f87-815f-4e78-bfb8-885f23cb4d70 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:16.974586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce6f87-815f-4e78-bfb8-885f23cb4d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:16.981759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 162/321] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:16.991891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-03ce6f87-815f-4e78-bfb8-885f23cb4d70 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:16.993066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-03ce6f87-815f-4e78-bfb8-885f23cb4d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:17.001821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c60f0-564a-4439-a6dc-436e01bcb02f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:17.018274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:17.066893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 160/322] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:06:17.089266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce6f87-815f-4e78-bfb8-885f23cb4d70 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:17.110034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c60f0-564a-4439-a6dc-436e01bcb02f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:17.149673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2361ed-ab58-4920-8132-ae0d4415458b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:17.150439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2361ed-ab58-4920-8132-ae0d4415458b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:17.150723 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2361ed-ab58-4920-8132-ae0d4415458b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:17.150997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2361ed-ab58-4920-8132-ae0d4415458b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:17.152177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2361ed-ab58-4920-8132-ae0d4415458b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:17.153635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2361ed-ab58-4920-8132-ae0d4415458b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:17.155369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 163/323] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:17.161098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2361ed-ab58-4920-8132-ae0d4415458b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:17.193982 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:17.205239 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce6f87-815f-4e78-bfb8-885f23cb4d70 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:17.241964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6a9e8-f8e6-4e1f-976a-fbc49cf2d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:17.242767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6a9e8-f8e6-4e1f-976a-fbc49cf2d15e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:17.243048 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6a9e8-f8e6-4e1f-976a-fbc49cf2d15e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:17.243319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6a9e8-f8e6-4e1f-976a-fbc49cf2d15e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:17.254471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6a9e8-f8e6-4e1f-976a-fbc49cf2d15e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1aa6a9e8-f8e6-4e1f-976a-fbc49cf2d15e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:17.262374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:17.263322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:17.264472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 161/324] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:17.269581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6a9e8-f8e6-4e1f-976a-fbc49cf2d15e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:17.275582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2361ed-ab58-4920-8132-ae0d4415458b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:17.290054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:17.315737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb503f-b71d-4c9b-ba74-c63b8e03f178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:17.316405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb503f-b71d-4c9b-ba74-c63b8e03f178 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:17.316680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb503f-b71d-4c9b-ba74-c63b8e03f178 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:17.317162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb503f-b71d-4c9b-ba74-c63b8e03f178 None admin] PATH_INFO: `/v3/groups` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:17.317634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb503f-b71d-4c9b-ba74-c63b8e03f178 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:17.319096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb503f-b71d-4c9b-ba74-c63b8e03f178 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:17.334528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 164/325] 10.222.0.42 () {60 vars in 1134 bytes} [Wed May 25 16:06:17 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:06:17.395444 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6a9e8-f8e6-4e1f-976a-fbc49cf2d15e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:17.443299 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab78ad4-1e12-4b7f-b7f1-d7ce37df023f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:17.444393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78ad4-1e12-4b7f-b7f1-d7ce37df023f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:17.444881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78ad4-1e12-4b7f-b7f1-d7ce37df023f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:17.445416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78ad4-1e12-4b7f-b7f1-d7ce37df023f None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:17.446422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab78ad4-1e12-4b7f-b7f1-d7ce37df023f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:17.446935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab78ad4-1e12-4b7f-b7f1-d7ce37df023f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:17.448719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab78ad4-1e12-4b7f-b7f1-d7ce37df023f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:17.459653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 162/326] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:17.465414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb503f-b71d-4c9b-ba74-c63b8e03f178 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:17.574698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:17.588207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb503f-b71d-4c9b-ba74-c63b8e03f178 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:17.625279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac103b-3ae5-4284-8083-f56ade207a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:17.626448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac103b-3ae5-4284-8083-f56ade207a9e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:17.626934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac103b-3ae5-4284-8083-f56ade207a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:17.627474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac103b-3ae5-4284-8083-f56ade207a9e None admin] PATH_INFO: `/v3/users/6e11df1b2fcb45cbb4a927e2dccf2b0d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:17.628876 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac103b-3ae5-4284-8083-f56ade207a9e None admin] RBAC: Authorizing `identity:get_user(user_id=6e11df1b2fcb45cbb4a927e2dccf2b0d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:17.630522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac103b-3ae5-4284-8083-f56ade207a9e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:17.632635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 165/327] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:17 2022] GET /identity/v3/users/6e11df1b2fcb45cbb4a927e2dccf2b0d => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:17.639094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab78ad4-1e12-4b7f-b7f1-d7ce37df023f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:17.659515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b4de-4716-4af1-bcdb-9fb8fc7fa30e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:17.660086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b4de-4716-4af1-bcdb-9fb8fc7fa30e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:17.660528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b4de-4716-4af1-bcdb-9fb8fc7fa30e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:17.662260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 166/328] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:17.667389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cb09b-2b67-4a4d-b282-acc4c3bc4c1b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:17.667680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cb09b-2b67-4a4d-b282-acc4c3bc4c1b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:17.667965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cb09b-2b67-4a4d-b282-acc4c3bc4c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:17.685580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-626cb09b-2b67-4a4d-b282-acc4c3bc4c1b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:17.686764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-626cb09b-2b67-4a4d-b282-acc4c3bc4c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:17.751529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:17.783062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 167/329] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:17.783570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab78ad4-1e12-4b7f-b7f1-d7ce37df023f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:17.806993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cb09b-2b67-4a4d-b282-acc4c3bc4c1b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:17.821412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcc460-d058-4aae-956c-d875dd1df651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:17.822585 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc460-d058-4aae-956c-d875dd1df651 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:17.823043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc460-d058-4aae-956c-d875dd1df651 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:17.823475 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc460-d058-4aae-956c-d875dd1df651 None admin] PATH_INFO: `/v3/system/users/6e11df1b2fcb45cbb4a927e2dccf2b0d/roles/a200b6cb596344e385e5f234488c6018` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:17.825176 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcc460-d058-4aae-956c-d875dd1df651 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6e11df1b2fcb45cbb4a927e2dccf2b0d, role_id=a200b6cb596344e385e5f234488c6018)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:17.827328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcc460-d058-4aae-956c-d875dd1df651 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:17.840687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 163/330] 10.222.0.42 () {58 vars in 1334 bytes} [Wed May 25 16:06:17 2022] PUT /identity/v3/system/users/6e11df1b2fcb45cbb4a927e2dccf2b0d/roles/a200b6cb596344e385e5f234488c6018 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:06:17.923677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:17.938426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cb09b-2b67-4a4d-b282-acc4c3bc4c1b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:17.981587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c54d54-bf43-47a9-951e-813ad123f379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:17.982563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c54d54-bf43-47a9-951e-813ad123f379 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:17.982943 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c54d54-bf43-47a9-951e-813ad123f379 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:17.983223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c54d54-bf43-47a9-951e-813ad123f379 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:17.995580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c54d54-bf43-47a9-951e-813ad123f379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-50c54d54-bf43-47a9-951e-813ad123f379 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:18.004503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:18.005551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:18.006587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:18.007251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 168/331] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:18.017786 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcc460-d058-4aae-956c-d875dd1df651 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:18.142277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcc460-d058-4aae-956c-d875dd1df651 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:18.182149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664defbe-c532-40f8-9ab7-d503481d235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:18.182839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664defbe-c532-40f8-9ab7-d503481d235a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:18.183122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664defbe-c532-40f8-9ab7-d503481d235a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:18.183400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664defbe-c532-40f8-9ab7-d503481d235a None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:18.184155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664defbe-c532-40f8-9ab7-d503481d235a None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:18.184465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664defbe-c532-40f8-9ab7-d503481d235a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:18.185978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664defbe-c532-40f8-9ab7-d503481d235a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:18.195358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 164/332] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:18.200418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c54d54-bf43-47a9-951e-813ad123f379 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:18.347570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c54d54-bf43-47a9-951e-813ad123f379 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:18.383584 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caebd8e-f4c9-4fa5-8806-a71eaae252f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:18.384463 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caebd8e-f4c9-4fa5-8806-a71eaae252f1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:18.384744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caebd8e-f4c9-4fa5-8806-a71eaae252f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:18.385015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caebd8e-f4c9-4fa5-8806-a71eaae252f1 None admin] PATH_INFO: `/v3/users/43709c4fbdc944198fd4e2005d1b0e6c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:18.386266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caebd8e-f4c9-4fa5-8806-a71eaae252f1 None admin] RBAC: Authorizing `identity:get_user(user_id=43709c4fbdc944198fd4e2005d1b0e6c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:18.387714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caebd8e-f4c9-4fa5-8806-a71eaae252f1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:18.390674 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 169/333] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:18 2022] GET /identity/v3/users/43709c4fbdc944198fd4e2005d1b0e6c => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:18.396018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664defbe-c532-40f8-9ab7-d503481d235a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:18.411946 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:18.549280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664defbe-c532-40f8-9ab7-d503481d235a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:18.597005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4ab12-51bb-4d93-a8fe-b891f5eb4ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:18.597687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4ab12-51bb-4d93-a8fe-b891f5eb4ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:18.597992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4ab12-51bb-4d93-a8fe-b891f5eb4ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:18.598274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4ab12-51bb-4d93-a8fe-b891f5eb4ec4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:18.599111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4ab12-51bb-4d93-a8fe-b891f5eb4ec4 None admin] RBAC: Adding query filter params (role.id=7f6dd280cf8941dcb8f79ecf88dbda78, user.id=43709c4fbdc944198fd4e2005d1b0e6c, scope.system=all) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:18.599421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4ab12-51bb-4d93-a8fe-b891f5eb4ec4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:18.600854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4ab12-51bb-4d93-a8fe-b891f5eb4ec4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:18.622146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 165/334] 10.222.0.42 () {56 vars in 1308 bytes} [Wed May 25 16:06:18 2022] GET /identity/v3/role_assignments?user.id=43709c4fbdc944198fd4e2005d1b0e6c&scope.system=all&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 510 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:20.826081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df579d-d0ab-4a48-afd2-130e0fa36068 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:20.826623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df579d-d0ab-4a48-afd2-130e0fa36068 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:20.826623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df579d-d0ab-4a48-afd2-130e0fa36068 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:20.828309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 170/335] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:20.833863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429e78b-345d-4889-b94f-9e7514ff1e3c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:20.834770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429e78b-345d-4889-b94f-9e7514ff1e3c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:20.835238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429e78b-345d-4889-b94f-9e7514ff1e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:20.856768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9429e78b-345d-4889-b94f-9e7514ff1e3c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:20.859196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9429e78b-345d-4889-b94f-9e7514ff1e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:20.938890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:20.941923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a30e8e-0348-48d8-b8c6-6e0f16e53364 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:20.942628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a30e8e-0348-48d8-b8c6-6e0f16e53364 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:20.943129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a30e8e-0348-48d8-b8c6-6e0f16e53364 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:20.944998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 171/336] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:20.950279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022cc20-4bd5-4eb0-8996-ea1b0089e5d2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:20.950471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 166/337] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:20.951426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022cc20-4bd5-4eb0-8996-ea1b0089e5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:20.951784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022cc20-4bd5-4eb0-8996-ea1b0089e5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:20.975139 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3022cc20-4bd5-4eb0-8996-ea1b0089e5d2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:20.976658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3022cc20-4bd5-4eb0-8996-ea1b0089e5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:20.979971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429e78b-345d-4889-b94f-9e7514ff1e3c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:20.992144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:21.129925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 172/338] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:21.179060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022cc20-4bd5-4eb0-8996-ea1b0089e5d2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:21.184419 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429e78b-345d-4889-b94f-9e7514ff1e3c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:21.237373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526ebee-49e2-4822-8f8c-53063d39bd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:21.238606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526ebee-49e2-4822-8f8c-53063d39bd69 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:21.239077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526ebee-49e2-4822-8f8c-53063d39bd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:21.239649 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526ebee-49e2-4822-8f8c-53063d39bd69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:21.260379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526ebee-49e2-4822-8f8c-53063d39bd69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0526ebee-49e2-4822-8f8c-53063d39bd69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:21.270142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:21.271274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:21.272253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:21.276688 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 167/339] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:21.278307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec4c3c-a684-4302-967f-52e43d08402b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:21.278757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec4c3c-a684-4302-967f-52e43d08402b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:21.280197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec4c3c-a684-4302-967f-52e43d08402b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:21.283280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 168/340] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:21.284621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec4c3c-a684-4302-967f-52e43d08402b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:21.384857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022cc20-4bd5-4eb0-8996-ea1b0089e5d2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:21.394501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:21.438617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca3fdf-02c6-487d-866e-13650fa8d83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:21.439859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca3fdf-02c6-487d-866e-13650fa8d83b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:21.440169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca3fdf-02c6-487d-866e-13650fa8d83b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:21.440716 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca3fdf-02c6-487d-866e-13650fa8d83b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:21.454468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca3fdf-02c6-487d-866e-13650fa8d83b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-55ca3fdf-02c6-487d-866e-13650fa8d83b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:21.463130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:21.467933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:21.468772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:21.469397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec4c3c-a684-4302-967f-52e43d08402b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:21.471692 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 173/341] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:06:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:21.473187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e5e88-f4b7-4197-a6de-21b9add10472 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:21.473727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e5e88-f4b7-4197-a6de-21b9add10472 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:21.474225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e5e88-f4b7-4197-a6de-21b9add10472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:21.489215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:21.497073 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c43e5e88-f4b7-4197-a6de-21b9add10472 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:21.499512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c43e5e88-f4b7-4197-a6de-21b9add10472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:21.516398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d32e2-9368-4a9c-a574-d289ca89c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:21.517089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d32e2-9368-4a9c-a574-d289ca89c077 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:21.517368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d32e2-9368-4a9c-a574-d289ca89c077 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:21.517643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d32e2-9368-4a9c-a574-d289ca89c077 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:21.519784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d32e2-9368-4a9c-a574-d289ca89c077 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:21.519784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d32e2-9368-4a9c-a574-d289ca89c077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:21.520201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d32e2-9368-4a9c-a574-d289ca89c077 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:21.532534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 169/342] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:21.534648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc8146-8348-4dcf-992f-1bcc95653bb1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:21.534946 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc8146-8348-4dcf-992f-1bcc95653bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:21.535223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc8146-8348-4dcf-992f-1bcc95653bb1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:21.536945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 170/343] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:21.537823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc8146-8348-4dcf-992f-1bcc95653bb1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:21.590649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 174/344] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:21.592438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35e7bc-4510-4d1c-9141-abf6ec361b74 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:21.593033 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35e7bc-4510-4d1c-9141-abf6ec361b74 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:21.593530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35e7bc-4510-4d1c-9141-abf6ec361b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:21.611650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bf35e7bc-4510-4d1c-9141-abf6ec361b74 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:21.613220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bf35e7bc-4510-4d1c-9141-abf6ec361b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:21.673808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc8146-8348-4dcf-992f-1bcc95653bb1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:21.691928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 175/345] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:21.693103 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35e7bc-4510-4d1c-9141-abf6ec361b74 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:21.706180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:21.740381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35df2f8-2cc1-4bd7-8735-47bfb972fa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:21.741071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35df2f8-2cc1-4bd7-8735-47bfb972fa83 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:21.741700 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35df2f8-2cc1-4bd7-8735-47bfb972fa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:21.742109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35df2f8-2cc1-4bd7-8735-47bfb972fa83 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:21.742799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35df2f8-2cc1-4bd7-8735-47bfb972fa83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:21.743109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35df2f8-2cc1-4bd7-8735-47bfb972fa83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:21.744504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35df2f8-2cc1-4bd7-8735-47bfb972fa83 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:21.754013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 171/346] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:21.754848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35df2f8-2cc1-4bd7-8735-47bfb972fa83 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:21.793456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:21.839958 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35e7bc-4510-4d1c-9141-abf6ec361b74 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:21.894685 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d63291-2d69-4920-9634-23c8c7dcae93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:21.895355 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63291-2d69-4920-9634-23c8c7dcae93 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:21.895643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63291-2d69-4920-9634-23c8c7dcae93 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:21.895923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63291-2d69-4920-9634-23c8c7dcae93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:21.896275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35df2f8-2cc1-4bd7-8735-47bfb972fa83 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:21.948544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569792ee-21df-4069-8b63-5fe8c1bdc4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:21.949433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569792ee-21df-4069-8b63-5fe8c1bdc4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:21.949742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569792ee-21df-4069-8b63-5fe8c1bdc4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:21.950030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569792ee-21df-4069-8b63-5fe8c1bdc4c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:21.962914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569792ee-21df-4069-8b63-5fe8c1bdc4c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-569792ee-21df-4069-8b63-5fe8c1bdc4c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:21.972798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:21.973882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:21.974768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 172/347] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:21.975423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd357ad-b3fb-45e6-8141-6b0785536c67 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:21.976238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd357ad-b3fb-45e6-8141-6b0785536c67 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:21.976238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd357ad-b3fb-45e6-8141-6b0785536c67 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:21.977532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 173/348] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:21.979257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd357ad-b3fb-45e6-8141-6b0785536c67 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:21.986545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d63291-2d69-4920-9634-23c8c7dcae93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:21.988345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d63291-2d69-4920-9634-23c8c7dcae93 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:22.032937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:22.084060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:22.124224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd357ad-b3fb-45e6-8141-6b0785536c67 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:22.136081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 176/349] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:22.137068 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d63291-2d69-4920-9634-23c8c7dcae93 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:22.178831 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0dff5-8823-45a7-be76-0674799378aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:22.179959 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0dff5-8823-45a7-be76-0674799378aa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:22.180223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0dff5-8823-45a7-be76-0674799378aa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:22.180500 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0dff5-8823-45a7-be76-0674799378aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:22.201815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0dff5-8823-45a7-be76-0674799378aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-97e0dff5-8823-45a7-be76-0674799378aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:22.209695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:22.210628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:22.211413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 174/350] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:22.212098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0dff5-8823-45a7-be76-0674799378aa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:22.268238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d63291-2d69-4920-9634-23c8c7dcae93 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:22.309276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f22bff-56f0-4c6f-a012-e15cad3b6204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:22.310324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22bff-56f0-4c6f-a012-e15cad3b6204 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:22.310661 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22bff-56f0-4c6f-a012-e15cad3b6204 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:22.311232 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22bff-56f0-4c6f-a012-e15cad3b6204 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:22.332484 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:22.338150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0dff5-8823-45a7-be76-0674799378aa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:22.359211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:22.391619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bb0c3-b232-4825-a823-de123ff25a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:22.392314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bb0c3-b232-4825-a823-de123ff25a6f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:22.392604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bb0c3-b232-4825-a823-de123ff25a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:22.392883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bb0c3-b232-4825-a823-de123ff25a6f None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:22.393644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bb0c3-b232-4825-a823-de123ff25a6f None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:22.394018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bb0c3-b232-4825-a823-de123ff25a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:22.395425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bb0c3-b232-4825-a823-de123ff25a6f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:22.405432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f22bff-56f0-4c6f-a012-e15cad3b6204 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:22.405603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 175/351] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:22.406918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71954258-8539-4940-86be-0e32a97880d5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:22.407046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f22bff-56f0-4c6f-a012-e15cad3b6204 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:22.407229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71954258-8539-4940-86be-0e32a97880d5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:22.407495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71954258-8539-4940-86be-0e32a97880d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:22.433852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-71954258-8539-4940-86be-0e32a97880d5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:22.436206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-71954258-8539-4940-86be-0e32a97880d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:22.515218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 176/352] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:22.517641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71954258-8539-4940-86be-0e32a97880d5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:22.540625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 177/353] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:22.541464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f22bff-56f0-4c6f-a012-e15cad3b6204 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:22.623297 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:22.638006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:22.651270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71954258-8539-4940-86be-0e32a97880d5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:22.664130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f22bff-56f0-4c6f-a012-e15cad3b6204 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:22.690566 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35829d-3cdb-4dd5-9df5-09546eae612d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:22.691435 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35829d-3cdb-4dd5-9df5-09546eae612d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:22.691714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35829d-3cdb-4dd5-9df5-09546eae612d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:22.691982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35829d-3cdb-4dd5-9df5-09546eae612d None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:22.693168 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35829d-3cdb-4dd5-9df5-09546eae612d None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:22.694636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35829d-3cdb-4dd5-9df5-09546eae612d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:22.696685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 177/354] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:22 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:22.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35829d-3cdb-4dd5-9df5-09546eae612d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:22.711260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b08c3c-a8dd-457b-9bf4-23623ff2d31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:22.711942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08c3c-a8dd-457b-9bf4-23623ff2d31e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:22.712223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08c3c-a8dd-457b-9bf4-23623ff2d31e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:22.712502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08c3c-a8dd-457b-9bf4-23623ff2d31e None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:22.713259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b08c3c-a8dd-457b-9bf4-23623ff2d31e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:22.713570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b08c3c-a8dd-457b-9bf4-23623ff2d31e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:22.715000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b08c3c-a8dd-457b-9bf4-23623ff2d31e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:22.725276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 178/355] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:22.726190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b08c3c-a8dd-457b-9bf4-23623ff2d31e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:22.813895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35829d-3cdb-4dd5-9df5-09546eae612d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:22.838978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b08c3c-a8dd-457b-9bf4-23623ff2d31e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:22.852340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eabe42-512c-4bf1-a6b5-22d824bd6874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:22.853012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabe42-512c-4bf1-a6b5-22d824bd6874 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:22.853529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabe42-512c-4bf1-a6b5-22d824bd6874 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:22.853529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabe42-512c-4bf1-a6b5-22d824bd6874 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:22.870375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:22.885757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:22.889384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520f683-ccb9-4bfa-99ef-7a2bf1f58f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:22.890580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520f683-ccb9-4bfa-99ef-7a2bf1f58f9f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:22.890873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520f683-ccb9-4bfa-99ef-7a2bf1f58f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:22.891155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520f683-ccb9-4bfa-99ef-7a2bf1f58f9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:22.903234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520f683-ccb9-4bfa-99ef-7a2bf1f58f9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a520f683-ccb9-4bfa-99ef-7a2bf1f58f9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:22.912860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:22.914018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 179/356] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:06:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:22.914831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520f683-ccb9-4bfa-99ef-7a2bf1f58f9f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:22.947460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eabe42-512c-4bf1-a6b5-22d824bd6874 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:22.949832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eabe42-512c-4bf1-a6b5-22d824bd6874 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:23.037682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520f683-ccb9-4bfa-99ef-7a2bf1f58f9f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:23.078312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 178/357] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:23.080660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc11fb-4cd8-4e35-887f-9d261e35b1dc None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.080951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc11fb-4cd8-4e35-887f-9d261e35b1dc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.081230 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc11fb-4cd8-4e35-887f-9d261e35b1dc None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.083627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 179/358] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:23.083787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc11fb-4cd8-4e35-887f-9d261e35b1dc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:23.088224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7d773-d099-48b6-8018-617ebe8491a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:23.089103 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7d773-d099-48b6-8018-617ebe8491a6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.089487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7d773-d099-48b6-8018-617ebe8491a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.089801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7d773-d099-48b6-8018-617ebe8491a6 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.091038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7d773-d099-48b6-8018-617ebe8491a6 None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:23.092574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7d773-d099-48b6-8018-617ebe8491a6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:23.094553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 180/359] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:22 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:23.095189 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7d773-d099-48b6-8018-617ebe8491a6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:23.161793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:23.164020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:23.217220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc11fb-4cd8-4e35-887f-9d261e35b1dc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:23.221369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7d773-d099-48b6-8018-617ebe8491a6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:23.260170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fa15d-678b-49c4-aa5c-f3c4eedcf876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:23.261037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fa15d-678b-49c4-aa5c-f3c4eedcf876 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.261315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fa15d-678b-49c4-aa5c-f3c4eedcf876 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.261638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fa15d-678b-49c4-aa5c-f3c4eedcf876 None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.262475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4748f-035f-4186-b398-14997ef1b8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:23.263436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4748f-035f-4186-b398-14997ef1b8ea None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.263719 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4748f-035f-4186-b398-14997ef1b8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.264108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4748f-035f-4186-b398-14997ef1b8ea None admin] PATH_INFO: `/v3/users/6e11df1b2fcb45cbb4a927e2dccf2b0d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.265343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4748f-035f-4186-b398-14997ef1b8ea None admin] RBAC: Authorizing `identity:get_user(user_id=6e11df1b2fcb45cbb4a927e2dccf2b0d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:23.266912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4748f-035f-4186-b398-14997ef1b8ea None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:23.268379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fa15d-678b-49c4-aa5c-f3c4eedcf876 None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:23.268570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 181/360] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:06:23 2022] GET /identity/v3/users/6e11df1b2fcb45cbb4a927e2dccf2b0d => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:23.269338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4748f-035f-4186-b398-14997ef1b8ea None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:23.269827 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fa15d-678b-49c4-aa5c-f3c4eedcf876 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:23.271656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 180/361] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:23 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:23.272266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fa15d-678b-49c4-aa5c-f3c4eedcf876 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:23.398006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fa15d-678b-49c4-aa5c-f3c4eedcf876 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:23.398761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4748f-035f-4186-b398-14997ef1b8ea None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:23.422280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:23.423114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:23.437357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba89e9-fc3d-49e2-9ecc-abe0d8c8583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:23.437738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ef1c9-7c31-4a96-9a9a-82d953b7f23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:23.438270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba89e9-fc3d-49e2-9ecc-abe0d8c8583d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.438417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ef1c9-7c31-4a96-9a9a-82d953b7f23b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.438555 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba89e9-fc3d-49e2-9ecc-abe0d8c8583d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.438701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ef1c9-7c31-4a96-9a9a-82d953b7f23b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.438825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba89e9-fc3d-49e2-9ecc-abe0d8c8583d None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.438974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ef1c9-7c31-4a96-9a9a-82d953b7f23b None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.439733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ef1c9-7c31-4a96-9a9a-82d953b7f23b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:23.439990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba89e9-fc3d-49e2-9ecc-abe0d8c8583d None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:23.440077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ef1c9-7c31-4a96-9a9a-82d953b7f23b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:23.441380 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ef1c9-7c31-4a96-9a9a-82d953b7f23b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:23.441479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba89e9-fc3d-49e2-9ecc-abe0d8c8583d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:23.443196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 181/362] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:23 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:23.447748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f23e5-d2ab-4650-b34f-55569d972272 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.448034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f23e5-d2ab-4650-b34f-55569d972272 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.448312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f23e5-d2ab-4650-b34f-55569d972272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.449762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 182/363] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:23.450563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ef1c9-7c31-4a96-9a9a-82d953b7f23b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:23.466799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fc5f23e5-d2ab-4650-b34f-55569d972272 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:23.468142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fc5f23e5-d2ab-4650-b34f-55569d972272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:23.534890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 182/364] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:23.535731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f23e5-d2ab-4650-b34f-55569d972272 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:23.555638 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ef1c9-7c31-4a96-9a9a-82d953b7f23b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:23.593849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340f332-2745-4856-b786-0c6026515312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:23.594718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340f332-2745-4856-b786-0c6026515312 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.595001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340f332-2745-4856-b786-0c6026515312 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.595279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340f332-2745-4856-b786-0c6026515312 None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.596440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340f332-2745-4856-b786-0c6026515312 None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:23.598262 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340f332-2745-4856-b786-0c6026515312 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:23.599892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 183/365] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:23 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:23.603315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340f332-2745-4856-b786-0c6026515312 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:23.643290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f23e5-d2ab-4650-b34f-55569d972272 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:23.656294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:23.664806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:23.680102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095518-a3b6-4a1d-8f2c-5edf792e6fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:23.680757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095518-a3b6-4a1d-8f2c-5edf792e6fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.681033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095518-a3b6-4a1d-8f2c-5edf792e6fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.681317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095518-a3b6-4a1d-8f2c-5edf792e6fe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.682489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57095518-a3b6-4a1d-8f2c-5edf792e6fe7 None admin] RBAC: Adding query filter params (role.id=a200b6cb596344e385e5f234488c6018, user.id=6e11df1b2fcb45cbb4a927e2dccf2b0d, scope.system=all) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:23.683044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57095518-a3b6-4a1d-8f2c-5edf792e6fe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:23.684577 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57095518-a3b6-4a1d-8f2c-5edf792e6fe7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:23.696982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 183/366] 10.222.0.42 () {56 vars in 1308 bytes} [Wed May 25 16:06:23 2022] GET /identity/v3/role_assignments?user.id=6e11df1b2fcb45cbb4a927e2dccf2b0d&scope.system=all&role.id=a200b6cb596344e385e5f234488c6018 => generated 510 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:23.697802 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095518-a3b6-4a1d-8f2c-5edf792e6fe7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:23.706481 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340f332-2745-4856-b786-0c6026515312 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:23.745558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb52dd-98bc-49f7-9114-e68a12a3baad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:23.746252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb52dd-98bc-49f7-9114-e68a12a3baad None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.746535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb52dd-98bc-49f7-9114-e68a12a3baad None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.746814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb52dd-98bc-49f7-9114-e68a12a3baad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.747791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb52dd-98bc-49f7-9114-e68a12a3baad None admin] RBAC: Adding query filter params (group.id=0244ce9c37c24738a2035de27f780da7, role.id=7f6dd280cf8941dcb8f79ecf88dbda78, scope.project.id=82df840b0bc44187a1a9db5c2a668847) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:23.748690 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb52dd-98bc-49f7-9114-e68a12a3baad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:23.750632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb52dd-98bc-49f7-9114-e68a12a3baad None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:23.756877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 184/367] 10.222.0.42 () {56 vars in 1376 bytes} [Wed May 25 16:06:23 2022] GET /identity/v3/role_assignments?group.id=0244ce9c37c24738a2035de27f780da7&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:23.757798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb52dd-98bc-49f7-9114-e68a12a3baad None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:23.853279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095518-a3b6-4a1d-8f2c-5edf792e6fe7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:23.889574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb52dd-98bc-49f7-9114-e68a12a3baad None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:23.909663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf5b04-74ee-42cc-ba42-eb7b6353b8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:23.911202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf5b04-74ee-42cc-ba42-eb7b6353b8df None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.911687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf5b04-74ee-42cc-ba42-eb7b6353b8df None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.912323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf5b04-74ee-42cc-ba42-eb7b6353b8df None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.913098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:23.914259 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf5b04-74ee-42cc-ba42-eb7b6353b8df None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:23.915519 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf5b04-74ee-42cc-ba42-eb7b6353b8df None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:23.921912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 184/368] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:23 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:23.923301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf5b04-74ee-42cc-ba42-eb7b6353b8df None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:23.943511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e58ea-007f-46c3-b660-027b35b1522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:23.944190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e58ea-007f-46c3-b660-027b35b1522a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:23.944474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e58ea-007f-46c3-b660-027b35b1522a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:23.945229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e58ea-007f-46c3-b660-027b35b1522a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:23.988280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:24.035939 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e58ea-007f-46c3-b660-027b35b1522a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:24.038374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e58ea-007f-46c3-b660-027b35b1522a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:24.077852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf5b04-74ee-42cc-ba42-eb7b6353b8df None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:24.120576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff3153-0c58-43be-a7ba-11f78ef9d293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:24.121808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3153-0c58-43be-a7ba-11f78ef9d293 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:24.122547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3153-0c58-43be-a7ba-11f78ef9d293 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:24.123025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3153-0c58-43be-a7ba-11f78ef9d293 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:24.137065 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff3153-0c58-43be-a7ba-11f78ef9d293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-16ff3153-0c58-43be-a7ba-11f78ef9d293 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:24.146675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:24.148422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 185/369] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:06:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:24.149083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff3153-0c58-43be-a7ba-11f78ef9d293 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:24.184155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 185/370] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:24.184155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e58ea-007f-46c3-b660-027b35b1522a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:24.210271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:24.292784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff3153-0c58-43be-a7ba-11f78ef9d293 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:24.301264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:24.321619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e58ea-007f-46c3-b660-027b35b1522a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:24.339238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eea7f7-b709-4793-a171-3159c543f9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:24.339908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea7f7-b709-4793-a171-3159c543f9ca None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:24.340196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea7f7-b709-4793-a171-3159c543f9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:24.340474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea7f7-b709-4793-a171-3159c543f9ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:24.341737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eea7f7-b709-4793-a171-3159c543f9ca None admin] RBAC: Adding query filter params (group.id=0244ce9c37c24738a2035de27f780da7, role.id=e7180b44fc2945568dfb83dfaf8b5c79, scope.project.id=82df840b0bc44187a1a9db5c2a668847) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:24.342123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eea7f7-b709-4793-a171-3159c543f9ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:24.343588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eea7f7-b709-4793-a171-3159c543f9ca None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:24.357762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 186/371] 10.222.0.42 () {56 vars in 1376 bytes} [Wed May 25 16:06:24 2022] GET /identity/v3/role_assignments?group.id=0244ce9c37c24738a2035de27f780da7&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=e7180b44fc2945568dfb83dfaf8b5c79 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:24.357762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eea7f7-b709-4793-a171-3159c543f9ca None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:24.374221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9985b1-481d-4f90-92da-a475661af048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:24.375001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9985b1-481d-4f90-92da-a475661af048 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:24.375357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9985b1-481d-4f90-92da-a475661af048 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:24.375711 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9985b1-481d-4f90-92da-a475661af048 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:24.376651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9985b1-481d-4f90-92da-a475661af048 None admin] RBAC: Adding query filter params (group.id=0244ce9c37c24738a2035de27f780da7, role.id=7f6dd280cf8941dcb8f79ecf88dbda78, scope.project.id=affcd8ec317b49faaaa94fdb779a9948) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:24.377079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9985b1-481d-4f90-92da-a475661af048 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:24.378683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9985b1-481d-4f90-92da-a475661af048 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:24.385339 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 186/372] 10.222.0.42 () {56 vars in 1376 bytes} [Wed May 25 16:06:24 2022] GET /identity/v3/role_assignments?group.id=0244ce9c37c24738a2035de27f780da7&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:24.386355 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9985b1-481d-4f90-92da-a475661af048 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:24.531538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:24.547330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eea7f7-b709-4793-a171-3159c543f9ca None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:24.577809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9985b1-481d-4f90-92da-a475661af048 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:24.634700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c247efe-9345-473b-b9ce-dbc203dbea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:24.635866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c247efe-9345-473b-b9ce-dbc203dbea13 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:24.636314 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c247efe-9345-473b-b9ce-dbc203dbea13 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:24.636739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c247efe-9345-473b-b9ce-dbc203dbea13 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:24.638957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c247efe-9345-473b-b9ce-dbc203dbea13 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:24.640799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c247efe-9345-473b-b9ce-dbc203dbea13 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:24.643123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 187/373] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:24 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:24.649460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6288d57-f836-474d-92e2-1540394a87cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:24.650129 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6288d57-f836-474d-92e2-1540394a87cd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:24.650427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6288d57-f836-474d-92e2-1540394a87cd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:24.650706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6288d57-f836-474d-92e2-1540394a87cd None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:24.651471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6288d57-f836-474d-92e2-1540394a87cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:24.651779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6288d57-f836-474d-92e2-1540394a87cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:24.652928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c247efe-9345-473b-b9ce-dbc203dbea13 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:24.653171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6288d57-f836-474d-92e2-1540394a87cd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:24.667654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 187/374] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:24.673863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6288d57-f836-474d-92e2-1540394a87cd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:24.711036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:24.777470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c247efe-9345-473b-b9ce-dbc203dbea13 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:24.815984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833b926-128f-4b8c-a116-ea78a54025ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:24.817195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833b926-128f-4b8c-a116-ea78a54025ba None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:24.817658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833b926-128f-4b8c-a116-ea78a54025ba None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:24.818168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833b926-128f-4b8c-a116-ea78a54025ba None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:24.820084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833b926-128f-4b8c-a116-ea78a54025ba None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:24.821919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833b926-128f-4b8c-a116-ea78a54025ba None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:24.824086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 188/375] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:24 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:24.830321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833b926-128f-4b8c-a116-ea78a54025ba None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:24.886148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:24.915874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6288d57-f836-474d-92e2-1540394a87cd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:24.959294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833b926-128f-4b8c-a116-ea78a54025ba None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:25.001695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28548c4-8922-492a-93c0-03f85f455590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:25.002394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28548c4-8922-492a-93c0-03f85f455590 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:25.002796 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28548c4-8922-492a-93c0-03f85f455590 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:25.003430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28548c4-8922-492a-93c0-03f85f455590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:25.004719 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28548c4-8922-492a-93c0-03f85f455590 None admin] RBAC: Adding query filter params (group.id=0244ce9c37c24738a2035de27f780da7, role.id=e7180b44fc2945568dfb83dfaf8b5c79, scope.project.id=affcd8ec317b49faaaa94fdb779a9948) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:25.005382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28548c4-8922-492a-93c0-03f85f455590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:25.008116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28548c4-8922-492a-93c0-03f85f455590 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:25.015190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 189/376] 10.222.0.42 () {56 vars in 1376 bytes} [Wed May 25 16:06:24 2022] GET /identity/v3/role_assignments?group.id=0244ce9c37c24738a2035de27f780da7&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=e7180b44fc2945568dfb83dfaf8b5c79 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:25.042674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edd839-8582-4e03-aad5-0f267c2d7e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:25.043454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edd839-8582-4e03-aad5-0f267c2d7e0c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:25.043742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edd839-8582-4e03-aad5-0f267c2d7e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:25.044020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edd839-8582-4e03-aad5-0f267c2d7e0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:25.136473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6edd839-8582-4e03-aad5-0f267c2d7e0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:25.137999 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6edd839-8582-4e03-aad5-0f267c2d7e0c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:25.194698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:25.290987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 188/377] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:25.297215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28548c4-8922-492a-93c0-03f85f455590 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:25.440698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:25.473587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28548c4-8922-492a-93c0-03f85f455590 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:25.515941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a2227-9a75-4ddb-971a-3a4074ed0127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:25.516806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a2227-9a75-4ddb-971a-3a4074ed0127 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:25.517084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a2227-9a75-4ddb-971a-3a4074ed0127 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:25.517357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a2227-9a75-4ddb-971a-3a4074ed0127 None admin] PATH_INFO: `/v3/projects/55f82ef7ffb1469ea7582069b5da4934` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:25.518572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a2227-9a75-4ddb-971a-3a4074ed0127 None admin] RBAC: Authorizing `identity:get_project(project_id=55f82ef7ffb1469ea7582069b5da4934)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:25.520013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a2227-9a75-4ddb-971a-3a4074ed0127 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:25.522123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 190/378] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:25 2022] GET /identity/v3/projects/55f82ef7ffb1469ea7582069b5da4934 => generated 344 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:25.527203 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edd839-8582-4e03-aad5-0f267c2d7e0c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:25.684485 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edd839-8582-4e03-aad5-0f267c2d7e0c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:25.727480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:25.735794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eac6d-2cd1-4543-9933-d28945748b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:25.736945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eac6d-2cd1-4543-9933-d28945748b13 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:25.737432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eac6d-2cd1-4543-9933-d28945748b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:25.737960 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eac6d-2cd1-4543-9933-d28945748b13 None admin] PATH_INFO: `/v3/groups/0b5721eb211841ccb2b9ede53a4aa1fc` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:25.743316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eac6d-2cd1-4543-9933-d28945748b13 None admin] RBAC: Authorizing `identity:get_group(group_id=0b5721eb211841ccb2b9ede53a4aa1fc)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:25.744993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eac6d-2cd1-4543-9933-d28945748b13 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:25.747235 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 189/379] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:25 2022] GET /identity/v3/groups/0b5721eb211841ccb2b9ede53a4aa1fc => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:25.752724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a2227-9a75-4ddb-971a-3a4074ed0127 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:25.915478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a2227-9a75-4ddb-971a-3a4074ed0127 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:25.966601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e98eb-a462-4a18-b6e2-0b597057e57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:25.967495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e98eb-a462-4a18-b6e2-0b597057e57f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:25.967779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e98eb-a462-4a18-b6e2-0b597057e57f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:25.968052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e98eb-a462-4a18-b6e2-0b597057e57f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:25.968890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e98eb-a462-4a18-b6e2-0b597057e57f None admin] RBAC: Adding query filter params (group.id=0b5721eb211841ccb2b9ede53a4aa1fc, role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=55f82ef7ffb1469ea7582069b5da4934) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:25.969201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e98eb-a462-4a18-b6e2-0b597057e57f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:25.970697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e98eb-a462-4a18-b6e2-0b597057e57f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:25.983003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 191/380] 10.222.0.42 () {56 vars in 1376 bytes} [Wed May 25 16:06:25 2022] GET /identity/v3/role_assignments?group.id=0b5721eb211841ccb2b9ede53a4aa1fc&scope.project.id=55f82ef7ffb1469ea7582069b5da4934&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:26.269188 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41e2d2-9fe1-4b00-be82-a49dd3dd0c06 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:26.270097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41e2d2-9fe1-4b00-be82-a49dd3dd0c06 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:26.270554 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41e2d2-9fe1-4b00-be82-a49dd3dd0c06 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:26.272112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 190/381] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:26.277620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16e847-5ec9-4c69-b823-9acfd2dee09e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:26.277942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16e847-5ec9-4c69-b823-9acfd2dee09e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:26.278430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16e847-5ec9-4c69-b823-9acfd2dee09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:26.297392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ca16e847-5ec9-4c69-b823-9acfd2dee09e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:26.298912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ca16e847-5ec9-4c69-b823-9acfd2dee09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:26.308790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:26.395541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 192/382] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:26.430656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41e2d2-9fe1-4b00-be82-a49dd3dd0c06 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:26.607698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41e2d2-9fe1-4b00-be82-a49dd3dd0c06 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:26.723004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3ab34-6da4-4788-b3ae-43c3a8b02fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:26.723829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3ab34-6da4-4788-b3ae-43c3a8b02fde None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:26.724117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3ab34-6da4-4788-b3ae-43c3a8b02fde None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:26.724398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3ab34-6da4-4788-b3ae-43c3a8b02fde None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:26.749157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3ab34-6da4-4788-b3ae-43c3a8b02fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e8f3ab34-6da4-4788-b3ae-43c3a8b02fde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:26.760323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:26.761220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:26.762183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 191/383] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:26.768283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16e847-5ec9-4c69-b823-9acfd2dee09e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:26.898370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16e847-5ec9-4c69-b823-9acfd2dee09e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:26.937092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f021e-daf6-456c-b49d-161a2ef1309d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:26.937791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f021e-daf6-456c-b49d-161a2ef1309d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:26.941737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f021e-daf6-456c-b49d-161a2ef1309d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:26.941737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f021e-daf6-456c-b49d-161a2ef1309d None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:26.941737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f021e-daf6-456c-b49d-161a2ef1309d None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:26.941737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f021e-daf6-456c-b49d-161a2ef1309d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:26.945737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f021e-daf6-456c-b49d-161a2ef1309d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:26.954829 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 193/384] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:26.961845 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3ab34-6da4-4788-b3ae-43c3a8b02fde None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:27.110766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:27.206096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3ab34-6da4-4788-b3ae-43c3a8b02fde None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:27.286448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c0cc1-97ab-42c5-b295-fd4954bb5f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:27.287339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c0cc1-97ab-42c5-b295-fd4954bb5f9b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:27.287629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c0cc1-97ab-42c5-b295-fd4954bb5f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:27.287913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c0cc1-97ab-42c5-b295-fd4954bb5f9b None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:27.289157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c0cc1-97ab-42c5-b295-fd4954bb5f9b None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:27.290630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c0cc1-97ab-42c5-b295-fd4954bb5f9b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:27.292591 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 192/385] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:26 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:27.298601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f021e-daf6-456c-b49d-161a2ef1309d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:27.308558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:27.375344 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d6242-6c6a-472b-87af-479a007957e1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:27.375664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d6242-6c6a-472b-87af-479a007957e1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:27.375922 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d6242-6c6a-472b-87af-479a007957e1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:27.377315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 193/386] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:27.382997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c2779-f6fc-4f41-85c0-a680dded3cf2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:27.383290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c2779-f6fc-4f41-85c0-a680dded3cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:27.383568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c2779-f6fc-4f41-85c0-a680dded3cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:27.409032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7f4c2779-f6fc-4f41-85c0-a680dded3cf2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:27.413340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7f4c2779-f6fc-4f41-85c0-a680dded3cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:27.496633 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f021e-daf6-456c-b49d-161a2ef1309d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:27.539497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 194/387] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:27.540831 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd339649-ba2d-41c3-8955-922ea8e6f873 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:27.541115 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd339649-ba2d-41c3-8955-922ea8e6f873 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:27.541495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd339649-ba2d-41c3-8955-922ea8e6f873 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:27.543160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 195/388] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:27.549785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a37d6-5b9a-47ed-b33d-4621ede513be None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:27.550071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a37d6-5b9a-47ed-b33d-4621ede513be None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:27.550347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a37d6-5b9a-47ed-b33d-4621ede513be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:27.558186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109841f7-bdc4-4484-80c4-72d0ed337c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:27.559076 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109841f7-bdc4-4484-80c4-72d0ed337c98 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:27.559359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109841f7-bdc4-4484-80c4-72d0ed337c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:27.559636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109841f7-bdc4-4484-80c4-72d0ed337c98 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:27.561891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109841f7-bdc4-4484-80c4-72d0ed337c98 None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:27.563375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109841f7-bdc4-4484-80c4-72d0ed337c98 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:27.571428 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 194/389] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:27 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:27.571428 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109841f7-bdc4-4484-80c4-72d0ed337c98 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:27.582417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-dc1a37d6-5b9a-47ed-b33d-4621ede513be None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:27.583720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dc1a37d6-5b9a-47ed-b33d-4621ede513be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:27.640302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:27.669858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 196/390] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:06:27.670945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a37d6-5b9a-47ed-b33d-4621ede513be None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:27.672496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:27.713270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109841f7-bdc4-4484-80c4-72d0ed337c98 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:27.756046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcaaf8-c90e-469f-832a-f6b6aa2238d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:27.757022 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcaaf8-c90e-469f-832a-f6b6aa2238d7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:27.757409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcaaf8-c90e-469f-832a-f6b6aa2238d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:27.757854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcaaf8-c90e-469f-832a-f6b6aa2238d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:27.770618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dcaaf8-c90e-469f-832a-f6b6aa2238d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f8dcaaf8-c90e-469f-832a-f6b6aa2238d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:27.779961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:27.780916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:27.781703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 195/391] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:27.787642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcaaf8-c90e-469f-832a-f6b6aa2238d7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:27.809404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a37d6-5b9a-47ed-b33d-4621ede513be None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:27.856374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acec8a-aeb5-46bd-97b3-4acaa388503d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:27.857979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acec8a-aeb5-46bd-97b3-4acaa388503d None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:27.858271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acec8a-aeb5-46bd-97b3-4acaa388503d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:27.858952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acec8a-aeb5-46bd-97b3-4acaa388503d None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847/groups/0244ce9c37c24738a2035de27f780da7/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:27.861561 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acec8a-aeb5-46bd-97b3-4acaa388503d None admin] RBAC: Authorizing `identity:create_grant(project_id=82df840b0bc44187a1a9db5c2a668847, group_id=0244ce9c37c24738a2035de27f780da7, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:27.863745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acec8a-aeb5-46bd-97b3-4acaa388503d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:27.880297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 197/392] 10.222.0.42 () {58 vars in 1442 bytes} [Wed May 25 16:06:27 2022] PUT /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847/groups/0244ce9c37c24738a2035de27f780da7/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:06:27.880930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acec8a-aeb5-46bd-97b3-4acaa388503d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:27.916631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcaaf8-c90e-469f-832a-f6b6aa2238d7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:27.956815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:27.966148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87271c43-7ebc-4efc-9012-17c4533c54ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:27.968159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87271c43-7ebc-4efc-9012-17c4533c54ea None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:27.968572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87271c43-7ebc-4efc-9012-17c4533c54ea None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:27.968953 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87271c43-7ebc-4efc-9012-17c4533c54ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:27.971840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:27.982701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87271c43-7ebc-4efc-9012-17c4533c54ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-87271c43-7ebc-4efc-9012-17c4533c54ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:27.991593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:27.993008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:27.994811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:27.996267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 196/393] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:06:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:27.996402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732ef5e-d931-4820-9453-68c4aaca2464 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.001742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732ef5e-d931-4820-9453-68c4aaca2464 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.001742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732ef5e-d931-4820-9453-68c4aaca2464 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.001742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 197/394] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:28.001742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732ef5e-d931-4820-9453-68c4aaca2464 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:28.032671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acec8a-aeb5-46bd-97b3-4acaa388503d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:28.073104 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc2879-c5fc-4fd5-843a-99ca481185b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:28.073829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc2879-c5fc-4fd5-843a-99ca481185b9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.074127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc2879-c5fc-4fd5-843a-99ca481185b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.074415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc2879-c5fc-4fd5-843a-99ca481185b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.075331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc2879-c5fc-4fd5-843a-99ca481185b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:28.075759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc2879-c5fc-4fd5-843a-99ca481185b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:28.077201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc2879-c5fc-4fd5-843a-99ca481185b9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:28.087449 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 198/395] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:28.089076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d30753-22dc-4067-8d0d-8865ddb173af None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.089371 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d30753-22dc-4067-8d0d-8865ddb173af None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.089654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d30753-22dc-4067-8d0d-8865ddb173af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.110810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a9d30753-22dc-4067-8d0d-8865ddb173af None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:28.112518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a9d30753-22dc-4067-8d0d-8865ddb173af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:28.127221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732ef5e-d931-4820-9453-68c4aaca2464 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:28.175727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33763b-6652-421f-a61b-35ee7b71c8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:28.176663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33763b-6652-421f-a61b-35ee7b71c8ea None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.177322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33763b-6652-421f-a61b-35ee7b71c8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.177769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33763b-6652-421f-a61b-35ee7b71c8ea None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.179430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33763b-6652-421f-a61b-35ee7b71c8ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:28.179891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33763b-6652-421f-a61b-35ee7b71c8ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:28.182075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33763b-6652-421f-a61b-35ee7b71c8ea None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:28.193699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 198/396] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:28.195548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33763b-6652-421f-a61b-35ee7b71c8ea None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:28.202568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 199/397] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:28.203572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d30753-22dc-4067-8d0d-8865ddb173af None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:28.264762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:28.269875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:28.330943 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33763b-6652-421f-a61b-35ee7b71c8ea None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:28.349276 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d30753-22dc-4067-8d0d-8865ddb173af None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:28.373112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94053a-5f4a-494f-bf5c-63831e336d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:28.374587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94053a-5f4a-494f-bf5c-63831e336d2c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.374877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94053a-5f4a-494f-bf5c-63831e336d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.375157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94053a-5f4a-494f-bf5c-63831e336d2c None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.376681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94053a-5f4a-494f-bf5c-63831e336d2c None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:28.378868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94053a-5f4a-494f-bf5c-63831e336d2c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:28.381789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 199/398] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:28 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:28.382883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94053a-5f4a-494f-bf5c-63831e336d2c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:28.398679 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7393d-881d-481e-a4b3-756ab24a6fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:28.399619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7393d-881d-481e-a4b3-756ab24a6fdc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.400256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7393d-881d-481e-a4b3-756ab24a6fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.400536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7393d-881d-481e-a4b3-756ab24a6fdc None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.402094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7393d-881d-481e-a4b3-756ab24a6fdc None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:28.404546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7393d-881d-481e-a4b3-756ab24a6fdc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:28.411200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 200/399] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:28 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:28.411200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7393d-881d-481e-a4b3-756ab24a6fdc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:28.525323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94053a-5f4a-494f-bf5c-63831e336d2c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:28.541189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7393d-881d-481e-a4b3-756ab24a6fdc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:28.544581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:28.559197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:28.574325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b3268-3a77-43c8-8464-7a9e996583f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:28.575272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3268-3a77-43c8-8464-7a9e996583f6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.575834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3268-3a77-43c8-8464-7a9e996583f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.576119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3268-3a77-43c8-8464-7a9e996583f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.585257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ee1d4-e31c-4194-8a31-ba983f7f1394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:28.586465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ee1d4-e31c-4194-8a31-ba983f7f1394 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.586850 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ee1d4-e31c-4194-8a31-ba983f7f1394 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.587225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ee1d4-e31c-4194-8a31-ba983f7f1394 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.588572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ee1d4-e31c-4194-8a31-ba983f7f1394 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:28.589320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b3268-3a77-43c8-8464-7a9e996583f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:28.590421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ee1d4-e31c-4194-8a31-ba983f7f1394 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:28.597781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 201/400] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:28 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:28.598741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ee1d4-e31c-4194-8a31-ba983f7f1394 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-769b3268-3a77-43c8-8464-7a9e996583f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:28.599570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:28.600637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:28.602312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 200/401] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:06:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:28.603392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b3268-3a77-43c8-8464-7a9e996583f6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:28.721166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ee1d4-e31c-4194-8a31-ba983f7f1394 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:28.732595 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b3268-3a77-43c8-8464-7a9e996583f6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:28.758008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efe7ab-683a-473e-9465-34b99cf65a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:28.758892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efe7ab-683a-473e-9465-34b99cf65a20 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.759177 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efe7ab-683a-473e-9465-34b99cf65a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.759454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efe7ab-683a-473e-9465-34b99cf65a20 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.760672 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efe7ab-683a-473e-9465-34b99cf65a20 None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:28.762175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efe7ab-683a-473e-9465-34b99cf65a20 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:28.765076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 202/402] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:28 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:28.765995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efe7ab-683a-473e-9465-34b99cf65a20 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:28.770205 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d091429-dd33-438c-9bf5-ebbcd78b8d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:28.771108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d091429-dd33-438c-9bf5-ebbcd78b8d49 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.771392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d091429-dd33-438c-9bf5-ebbcd78b8d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.771673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d091429-dd33-438c-9bf5-ebbcd78b8d49 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948/groups/0244ce9c37c24738a2035de27f780da7/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.773706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d091429-dd33-438c-9bf5-ebbcd78b8d49 None admin] RBAC: Authorizing `identity:create_grant(project_id=affcd8ec317b49faaaa94fdb779a9948, group_id=0244ce9c37c24738a2035de27f780da7, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:28.775199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d091429-dd33-438c-9bf5-ebbcd78b8d49 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:28.794196 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 201/403] 10.222.0.42 () {58 vars in 1442 bytes} [Wed May 25 16:06:28 2022] PUT /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948/groups/0244ce9c37c24738a2035de27f780da7/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:06:28.795737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d091429-dd33-438c-9bf5-ebbcd78b8d49 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:28.832640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:28.863987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:28.916442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efe7ab-683a-473e-9465-34b99cf65a20 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:28.936872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d091429-dd33-438c-9bf5-ebbcd78b8d49 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:28.961927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca65614-f4ae-46dc-a11e-562f82d7a687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:28.962634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca65614-f4ae-46dc-a11e-562f82d7a687 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.963112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca65614-f4ae-46dc-a11e-562f82d7a687 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.963401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca65614-f4ae-46dc-a11e-562f82d7a687 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.964689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca65614-f4ae-46dc-a11e-562f82d7a687 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:28.965001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca65614-f4ae-46dc-a11e-562f82d7a687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:28.967682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca65614-f4ae-46dc-a11e-562f82d7a687 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:28.978028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0f666-ff2f-449f-b7ff-385a1dd1a1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:28.979115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0f666-ff2f-449f-b7ff-385a1dd1a1e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.979504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0f666-ff2f-449f-b7ff-385a1dd1a1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.979881 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0f666-ff2f-449f-b7ff-385a1dd1a1e6 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847/groups/0244ce9c37c24738a2035de27f780da7/roles/e7180b44fc2945568dfb83dfaf8b5c79` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.980071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 203/404] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:28.981486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a27b28-0fd2-437f-8b28-bf60f53b45b6 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:28.982404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a27b28-0fd2-437f-8b28-bf60f53b45b6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:28.982713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a27b28-0fd2-437f-8b28-bf60f53b45b6 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:28.983727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0f666-ff2f-449f-b7ff-385a1dd1a1e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=82df840b0bc44187a1a9db5c2a668847, group_id=0244ce9c37c24738a2035de27f780da7, role_id=e7180b44fc2945568dfb83dfaf8b5c79)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:28.984412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 204/405] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:28.985511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0f666-ff2f-449f-b7ff-385a1dd1a1e6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:28.989325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a27b28-0fd2-437f-8b28-bf60f53b45b6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:29.007356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 202/406] 10.222.0.42 () {58 vars in 1442 bytes} [Wed May 25 16:06:28 2022] PUT /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847/groups/0244ce9c37c24738a2035de27f780da7/roles/e7180b44fc2945568dfb83dfaf8b5c79 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:06:29.009486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2bf6d-1399-4309-8578-71ed9b885906 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:29.009945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2bf6d-1399-4309-8578-71ed9b885906 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:29.010497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2bf6d-1399-4309-8578-71ed9b885906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:29.036227 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-39d2bf6d-1399-4309-8578-71ed9b885906 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:29.040087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-39d2bf6d-1399-4309-8578-71ed9b885906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:29.133856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 203/407] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:29.136560 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a27b28-0fd2-437f-8b28-bf60f53b45b6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:29.138731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:29.164483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2bf6d-1399-4309-8578-71ed9b885906 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:29.223781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb382ac-541f-4b31-b33e-0696c9f03d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:29.223781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb382ac-541f-4b31-b33e-0696c9f03d86 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:29.223781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb382ac-541f-4b31-b33e-0696c9f03d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:29.223781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb382ac-541f-4b31-b33e-0696c9f03d86 None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:29.224928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb382ac-541f-4b31-b33e-0696c9f03d86 None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:29.226546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb382ac-541f-4b31-b33e-0696c9f03d86 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:29.228348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 205/408] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:28 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:29.231261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:29.234473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb382ac-541f-4b31-b33e-0696c9f03d86 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:29.324921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2bf6d-1399-4309-8578-71ed9b885906 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:29.362852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb382ac-541f-4b31-b33e-0696c9f03d86 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:29.369458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95516ae-de1d-4b97-9491-58d4cae788d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:29.370312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95516ae-de1d-4b97-9491-58d4cae788d5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:29.370590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95516ae-de1d-4b97-9491-58d4cae788d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:29.370861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95516ae-de1d-4b97-9491-58d4cae788d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:29.386378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95516ae-de1d-4b97-9491-58d4cae788d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b95516ae-de1d-4b97-9491-58d4cae788d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:29.399378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:29.400352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:29.401149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 204/409] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:06:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:29.407554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95516ae-de1d-4b97-9491-58d4cae788d5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:29.424187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ca702-0bf7-44b2-bb38-df8e69838ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:29.426001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ca702-0bf7-44b2-bb38-df8e69838ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:29.426525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ca702-0bf7-44b2-bb38-df8e69838ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:29.427003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ca702-0bf7-44b2-bb38-df8e69838ca1 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:29.429413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ca702-0bf7-44b2-bb38-df8e69838ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:29.431131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ca702-0bf7-44b2-bb38-df8e69838ca1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:29.436428 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 206/410] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:29 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:29.441170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ca702-0bf7-44b2-bb38-df8e69838ca1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:29.521328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:29.574303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:29.588439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95516ae-de1d-4b97-9491-58d4cae788d5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:29.659444 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ca702-0bf7-44b2-bb38-df8e69838ca1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:29.661333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacb6ea-4b71-4f59-9a24-25550d9f7a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:29.662032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacb6ea-4b71-4f59-9a24-25550d9f7a15 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:29.662320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacb6ea-4b71-4f59-9a24-25550d9f7a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:29.662597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacb6ea-4b71-4f59-9a24-25550d9f7a15 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:29.663357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacb6ea-4b71-4f59-9a24-25550d9f7a15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:29.663666 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacb6ea-4b71-4f59-9a24-25550d9f7a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:29.665051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacb6ea-4b71-4f59-9a24-25550d9f7a15 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:29.683577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 205/411] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:29.689540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacb6ea-4b71-4f59-9a24-25550d9f7a15 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:29.734101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e68d0-72cb-449d-bf82-10054897ebd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:29.735054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e68d0-72cb-449d-bf82-10054897ebd8 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:29.735345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e68d0-72cb-449d-bf82-10054897ebd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:29.735620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e68d0-72cb-449d-bf82-10054897ebd8 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948/groups/0244ce9c37c24738a2035de27f780da7/roles/e7180b44fc2945568dfb83dfaf8b5c79` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:29.739841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e68d0-72cb-449d-bf82-10054897ebd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=affcd8ec317b49faaaa94fdb779a9948, group_id=0244ce9c37c24738a2035de27f780da7, role_id=e7180b44fc2945568dfb83dfaf8b5c79)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:29.741306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e68d0-72cb-449d-bf82-10054897ebd8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:29.767831 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 207/412] 10.222.0.42 () {58 vars in 1442 bytes} [Wed May 25 16:06:29 2022] PUT /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948/groups/0244ce9c37c24738a2035de27f780da7/roles/e7180b44fc2945568dfb83dfaf8b5c79 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:06:29.838295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacb6ea-4b71-4f59-9a24-25550d9f7a15 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:29.891550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd3b19-b2d3-4f61-b550-f36c87819d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:29.892443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd3b19-b2d3-4f61-b550-f36c87819d71 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:29.892730 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd3b19-b2d3-4f61-b550-f36c87819d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:29.893005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd3b19-b2d3-4f61-b550-f36c87819d71 None admin] PATH_INFO: `/v3/groups/0b5721eb211841ccb2b9ede53a4aa1fc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:29.898264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd3b19-b2d3-4f61-b550-f36c87819d71 None admin] RBAC: Authorizing `identity:get_group(group_id=0b5721eb211841ccb2b9ede53a4aa1fc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:29.899872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd3b19-b2d3-4f61-b550-f36c87819d71 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:29.901988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 206/413] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:29 2022] GET /identity/v3/groups/0b5721eb211841ccb2b9ede53a4aa1fc => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:29.907463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e68d0-72cb-449d-bf82-10054897ebd8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:30.030216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:30.038491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e68d0-72cb-449d-bf82-10054897ebd8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:30.077821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d8301-effa-4899-9c4f-1d3097955346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:30.078711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d8301-effa-4899-9c4f-1d3097955346 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:30.078992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d8301-effa-4899-9c4f-1d3097955346 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:30.079267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d8301-effa-4899-9c4f-1d3097955346 None admin] PATH_INFO: `/v3/projects/55f82ef7ffb1469ea7582069b5da4934` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:30.080495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d8301-effa-4899-9c4f-1d3097955346 None admin] RBAC: Authorizing `identity:get_project(project_id=55f82ef7ffb1469ea7582069b5da4934)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:30.082002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d8301-effa-4899-9c4f-1d3097955346 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:30.084066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 208/414] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:29 2022] GET /identity/v3/projects/55f82ef7ffb1469ea7582069b5da4934 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:30.089544 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd3b19-b2d3-4f61-b550-f36c87819d71 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:30.132906 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:30.235814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd3b19-b2d3-4f61-b550-f36c87819d71 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:30.285627 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2dedf-9e1f-4e0c-a5b0-c36eb1747b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:30.286886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2dedf-9e1f-4e0c-a5b0-c36eb1747b1d None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:30.287378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2dedf-9e1f-4e0c-a5b0-c36eb1747b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:30.287877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2dedf-9e1f-4e0c-a5b0-c36eb1747b1d None admin] PATH_INFO: `/v3/projects/55f82ef7ffb1469ea7582069b5da4934/groups/0b5721eb211841ccb2b9ede53a4aa1fc/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:30.290253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2dedf-9e1f-4e0c-a5b0-c36eb1747b1d None admin] RBAC: Authorizing `identity:create_grant(project_id=55f82ef7ffb1469ea7582069b5da4934, group_id=0b5721eb211841ccb2b9ede53a4aa1fc, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:30.291976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2dedf-9e1f-4e0c-a5b0-c36eb1747b1d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:30.315333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 207/415] 10.222.0.42 () {58 vars in 1442 bytes} [Wed May 25 16:06:30 2022] PUT /identity/v3/projects/55f82ef7ffb1469ea7582069b5da4934/groups/0b5721eb211841ccb2b9ede53a4aa1fc/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:06:30.377557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873671b-5c69-4183-87d0-e4efe7cb431b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:30.378138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873671b-5c69-4183-87d0-e4efe7cb431b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:30.378138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873671b-5c69-4183-87d0-e4efe7cb431b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:30.379907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 209/416] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:30.385974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc604a-2a45-4718-8f54-78a3e29c1949 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:30.386925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc604a-2a45-4718-8f54-78a3e29c1949 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:30.387446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc604a-2a45-4718-8f54-78a3e29c1949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:30.406127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-34dc604a-2a45-4718-8f54-78a3e29c1949 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:30.407697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-34dc604a-2a45-4718-8f54-78a3e29c1949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:30.483479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 208/417] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:30.515660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873671b-5c69-4183-87d0-e4efe7cb431b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:30.652671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873671b-5c69-4183-87d0-e4efe7cb431b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:30.693006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f706627-8080-4543-a4e2-05103fb84e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:30.694170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706627-8080-4543-a4e2-05103fb84e1b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:30.694593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706627-8080-4543-a4e2-05103fb84e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:30.695034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706627-8080-4543-a4e2-05103fb84e1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:30.706444 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f706627-8080-4543-a4e2-05103fb84e1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9f706627-8080-4543-a4e2-05103fb84e1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:30.714368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:30.715440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:30.716494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:30.718908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 210/418] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:30.724354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc604a-2a45-4718-8f54-78a3e29c1949 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:30.755193 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:30.870119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc604a-2a45-4718-8f54-78a3e29c1949 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:30.914542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b4b3a-ede5-4a29-a445-6f132efa8aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:30.915134 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b4b3a-ede5-4a29-a445-6f132efa8aec None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:30.915419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b4b3a-ede5-4a29-a445-6f132efa8aec None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:30.915694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b4b3a-ede5-4a29-a445-6f132efa8aec None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:30.916454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b4b3a-ede5-4a29-a445-6f132efa8aec None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:30.916758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b4b3a-ede5-4a29-a445-6f132efa8aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:30.918172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b4b3a-ede5-4a29-a445-6f132efa8aec None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:30.932226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 209/419] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:30.938000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f706627-8080-4543-a4e2-05103fb84e1b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:30.984635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:31.072412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f706627-8080-4543-a4e2-05103fb84e1b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:31.112888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca03de-7d6b-4b30-bc35-5fcb2bb36a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:31.113557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca03de-7d6b-4b30-bc35-5fcb2bb36a84 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:31.113880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca03de-7d6b-4b30-bc35-5fcb2bb36a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:31.114216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca03de-7d6b-4b30-bc35-5fcb2bb36a84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:31.194711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca03de-7d6b-4b30-bc35-5fcb2bb36a84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:31.196100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca03de-7d6b-4b30-bc35-5fcb2bb36a84 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:31.228636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:31.331407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 211/420] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:31.337069 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b4b3a-ede5-4a29-a445-6f132efa8aec None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:31.453552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:31.497574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b4b3a-ede5-4a29-a445-6f132efa8aec None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:31.539012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33bf17-60f0-44d1-af5b-322347b8c69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:31.539903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33bf17-60f0-44d1-af5b-322347b8c69a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:31.540190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33bf17-60f0-44d1-af5b-322347b8c69a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:31.540466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33bf17-60f0-44d1-af5b-322347b8c69a None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:31.541746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33bf17-60f0-44d1-af5b-322347b8c69a None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:31.543215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33bf17-60f0-44d1-af5b-322347b8c69a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:31.546103 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 210/421] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:31 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:31.552094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca03de-7d6b-4b30-bc35-5fcb2bb36a84 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:31.708400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca03de-7d6b-4b30-bc35-5fcb2bb36a84 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:31.748917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:31.751735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358578c-d9bc-4e92-997a-06f0fa2f77ce None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:31.752055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358578c-d9bc-4e92-997a-06f0fa2f77ce None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:31.752332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358578c-d9bc-4e92-997a-06f0fa2f77ce None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:31.756894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 211/422] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:31.759659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15632e8f-bd9b-41a6-9bac-fb0bd64b3ff8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:31.759953 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15632e8f-bd9b-41a6-9bac-fb0bd64b3ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:31.760233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15632e8f-bd9b-41a6-9bac-fb0bd64b3ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:31.763008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aababe9-583f-48c7-b9c5-4ee01c0efd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:31.763878 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aababe9-583f-48c7-b9c5-4ee01c0efd0f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:31.764163 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aababe9-583f-48c7-b9c5-4ee01c0efd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:31.764438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aababe9-583f-48c7-b9c5-4ee01c0efd0f None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:31.765649 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aababe9-583f-48c7-b9c5-4ee01c0efd0f None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:31.767126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aababe9-583f-48c7-b9c5-4ee01c0efd0f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:31.769080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 212/423] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:31 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:31.774640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aababe9-583f-48c7-b9c5-4ee01c0efd0f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:31.777583 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-15632e8f-bd9b-41a6-9bac-fb0bd64b3ff8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:31.779565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-15632e8f-bd9b-41a6-9bac-fb0bd64b3ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:31.867831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 212/424] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:31.895494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15632e8f-bd9b-41a6-9bac-fb0bd64b3ff8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:31.968666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aababe9-583f-48c7-b9c5-4ee01c0efd0f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:31.990998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:32.009548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101e21a-1304-4f51-a516-029663711c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:32.010246 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101e21a-1304-4f51-a516-029663711c9d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:32.010529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101e21a-1304-4f51-a516-029663711c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:32.010809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101e21a-1304-4f51-a516-029663711c9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:32.011637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101e21a-1304-4f51-a516-029663711c9d None admin] RBAC: Adding query filter params (group.id=0244ce9c37c24738a2035de27f780da7, role.id=7f6dd280cf8941dcb8f79ecf88dbda78, scope.project.id=82df840b0bc44187a1a9db5c2a668847) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:32.011950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101e21a-1304-4f51-a516-029663711c9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:32.014272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101e21a-1304-4f51-a516-029663711c9d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:32.021433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 213/425] 10.222.0.42 () {56 vars in 1376 bytes} [Wed May 25 16:06:31 2022] GET /identity/v3/role_assignments?group.id=0244ce9c37c24738a2035de27f780da7&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 611 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:32.022779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149900f-8161-4a83-aef0-82e9902beadf None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:32.023071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149900f-8161-4a83-aef0-82e9902beadf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:32.023351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149900f-8161-4a83-aef0-82e9902beadf None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:32.023803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15632e8f-bd9b-41a6-9bac-fb0bd64b3ff8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:32.024875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 214/426] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:32.034368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824a3c0-955f-496b-9538-930f084a77e6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:32.034791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824a3c0-955f-496b-9538-930f084a77e6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:32.035029 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824a3c0-955f-496b-9538-930f084a77e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:32.069265 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a824a3c0-955f-496b-9538-930f084a77e6 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:32.070915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a824a3c0-955f-496b-9538-930f084a77e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:32.103369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700cfca-99ef-47c1-970c-3cd71d650129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:32.104435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700cfca-99ef-47c1-970c-3cd71d650129 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:32.104685 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700cfca-99ef-47c1-970c-3cd71d650129 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:32.104960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700cfca-99ef-47c1-970c-3cd71d650129 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:32.127734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700cfca-99ef-47c1-970c-3cd71d650129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a700cfca-99ef-47c1-970c-3cd71d650129 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:32.139701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:32.140700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:32.141517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 213/427] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:32.148171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700cfca-99ef-47c1-970c-3cd71d650129 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:32.187038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:32.216871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 215/428] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:32.249582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824a3c0-955f-496b-9538-930f084a77e6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:32.333152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700cfca-99ef-47c1-970c-3cd71d650129 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:32.377937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c6ce4-8a5a-4857-9f57-0e5856c30bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:32.377937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c6ce4-8a5a-4857-9f57-0e5856c30bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:32.377937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c6ce4-8a5a-4857-9f57-0e5856c30bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:32.377937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c6ce4-8a5a-4857-9f57-0e5856c30bd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:32.380550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c6ce4-8a5a-4857-9f57-0e5856c30bd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:32.380550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c6ce4-8a5a-4857-9f57-0e5856c30bd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:32.380550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c6ce4-8a5a-4857-9f57-0e5856c30bd9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:32.383015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:32.387738 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824a3c0-955f-496b-9538-930f084a77e6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:32.392693 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 214/429] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:06:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:32.398177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c6ce4-8a5a-4857-9f57-0e5856c30bd9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:32.446130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea655bb4-7182-45fa-bde4-da5f029b5479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:32.447417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea655bb4-7182-45fa-bde4-da5f029b5479 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:32.447854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea655bb4-7182-45fa-bde4-da5f029b5479 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:32.450207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea655bb4-7182-45fa-bde4-da5f029b5479 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:32.468193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea655bb4-7182-45fa-bde4-da5f029b5479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ea655bb4-7182-45fa-bde4-da5f029b5479 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:32.476416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:32.477357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:32.478196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:32.479770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 216/430] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:06:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:32.489741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea655bb4-7182-45fa-bde4-da5f029b5479 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:32.545317 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c6ce4-8a5a-4857-9f57-0e5856c30bd9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:32.555438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:32.594407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c48861-74f7-431e-914c-af05419c1404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:32.595070 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c48861-74f7-431e-914c-af05419c1404 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:32.595356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c48861-74f7-431e-914c-af05419c1404 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:32.595635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c48861-74f7-431e-914c-af05419c1404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:32.620206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea655bb4-7182-45fa-bde4-da5f029b5479 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:32.660767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a1b41-2cb1-4ce5-9872-2f77bb05bce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:32.661885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a1b41-2cb1-4ce5-9872-2f77bb05bce0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:32.662439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a1b41-2cb1-4ce5-9872-2f77bb05bce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:32.662910 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a1b41-2cb1-4ce5-9872-2f77bb05bce0 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:32.663921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a1b41-2cb1-4ce5-9872-2f77bb05bce0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:32.664426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a1b41-2cb1-4ce5-9872-2f77bb05bce0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:32.666234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a1b41-2cb1-4ce5-9872-2f77bb05bce0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:32.674683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:32.677085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 217/431] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:32.683011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a1b41-2cb1-4ce5-9872-2f77bb05bce0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:32.683994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c48861-74f7-431e-914c-af05419c1404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:32.685760 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c48861-74f7-431e-914c-af05419c1404 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:32.812631 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a1b41-2cb1-4ce5-9872-2f77bb05bce0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:32.826861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 215/432] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 16:06:32.827351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:32.832309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c48861-74f7-431e-914c-af05419c1404 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:32.866511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e3918-6b36-4a25-a473-62ee699694c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:32.867181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3918-6b36-4a25-a473-62ee699694c9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:32.867467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3918-6b36-4a25-a473-62ee699694c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:32.867752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3918-6b36-4a25-a473-62ee699694c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:32.961179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e3918-6b36-4a25-a473-62ee699694c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:32.963743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e3918-6b36-4a25-a473-62ee699694c9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:32.974847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:32.984431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c48861-74f7-431e-914c-af05419c1404 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:33.023371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708953c-611e-467f-b4a4-07d07e3d6c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:33.024240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708953c-611e-467f-b4a4-07d07e3d6c80 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.024520 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708953c-611e-467f-b4a4-07d07e3d6c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.024791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708953c-611e-467f-b4a4-07d07e3d6c80 None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.026230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708953c-611e-467f-b4a4-07d07e3d6c80 None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:33.027680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708953c-611e-467f-b4a4-07d07e3d6c80 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:33.035687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 216/433] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:32 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:33.035687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e4b4a-16cb-4480-bed9-89fffde4eec2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.035687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e4b4a-16cb-4480-bed9-89fffde4eec2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.035687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e4b4a-16cb-4480-bed9-89fffde4eec2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.035687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 217/434] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:33.037211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e4b4a-16cb-4480-bed9-89fffde4eec2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:33.094445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 218/435] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 16:06:33.096170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f48592-c02c-44d7-a59c-d69b57e0c17c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.096518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f48592-c02c-44d7-a59c-d69b57e0c17c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.096861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f48592-c02c-44d7-a59c-d69b57e0c17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.114482 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d4f48592-c02c-44d7-a59c-d69b57e0c17c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:33.115754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d4f48592-c02c-44d7-a59c-d69b57e0c17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:33.118881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:33.179213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e4b4a-16cb-4480-bed9-89fffde4eec2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:33.202538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 219/436] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:33.203425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f48592-c02c-44d7-a59c-d69b57e0c17c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:33.220311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa930597-30a5-4cc4-90ad-62aa1aabb478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:33.221530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa930597-30a5-4cc4-90ad-62aa1aabb478 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.222030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa930597-30a5-4cc4-90ad-62aa1aabb478 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.222540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa930597-30a5-4cc4-90ad-62aa1aabb478 None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.223940 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa930597-30a5-4cc4-90ad-62aa1aabb478 None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:33.225539 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa930597-30a5-4cc4-90ad-62aa1aabb478 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:33.227930 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 218/437] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:33 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:33.235242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa930597-30a5-4cc4-90ad-62aa1aabb478 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:33.278039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:33.345118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f48592-c02c-44d7-a59c-d69b57e0c17c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:33.370573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa930597-30a5-4cc4-90ad-62aa1aabb478 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:33.387311 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c585f2b-6559-40d1-b144-3b2b8a822027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:33.388880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c585f2b-6559-40d1-b144-3b2b8a822027 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.389353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c585f2b-6559-40d1-b144-3b2b8a822027 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.389637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c585f2b-6559-40d1-b144-3b2b8a822027 None admin] PATH_INFO: `/v3/projects/82df840b0bc44187a1a9db5c2a668847` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.391009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c585f2b-6559-40d1-b144-3b2b8a822027 None admin] RBAC: Authorizing `identity:get_project(project_id=82df840b0bc44187a1a9db5c2a668847)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:33.392605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c585f2b-6559-40d1-b144-3b2b8a822027 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:33.394462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 220/438] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:33 2022] GET /identity/v3/projects/82df840b0bc44187a1a9db5c2a668847 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:33.395225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c585f2b-6559-40d1-b144-3b2b8a822027 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:33.407775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:33.413896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075cfdf9-f0f9-49fa-980f-d778570ba296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:33.415841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cfdf9-f0f9-49fa-980f-d778570ba296 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.416125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cfdf9-f0f9-49fa-980f-d778570ba296 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.416400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cfdf9-f0f9-49fa-980f-d778570ba296 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.418123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075cfdf9-f0f9-49fa-980f-d778570ba296 None admin] RBAC: Adding query filter params (group.id=0244ce9c37c24738a2035de27f780da7, role.id=7f6dd280cf8941dcb8f79ecf88dbda78, scope.project.id=affcd8ec317b49faaaa94fdb779a9948) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:33.419159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075cfdf9-f0f9-49fa-980f-d778570ba296 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:33.420662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075cfdf9-f0f9-49fa-980f-d778570ba296 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:33.427637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 219/439] 10.222.0.42 () {56 vars in 1376 bytes} [Wed May 25 16:06:33 2022] GET /identity/v3/role_assignments?group.id=0244ce9c37c24738a2035de27f780da7&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=7f6dd280cf8941dcb8f79ecf88dbda78 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:33.431267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163597a-b836-470b-9094-23fccba4253b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.431267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163597a-b836-470b-9094-23fccba4253b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.431267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163597a-b836-470b-9094-23fccba4253b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.432222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 220/440] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:33.434785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163597a-b836-470b-9094-23fccba4253b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:33.542881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c585f2b-6559-40d1-b144-3b2b8a822027 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:33.557296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:33.566194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163597a-b836-470b-9094-23fccba4253b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:33.594425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c1262-2ad4-45bb-8fe5-c1f32c530281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:33.596490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c1262-2ad4-45bb-8fe5-c1f32c530281 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.596490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c1262-2ad4-45bb-8fe5-c1f32c530281 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.596490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c1262-2ad4-45bb-8fe5-c1f32c530281 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.610367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c1262-2ad4-45bb-8fe5-c1f32c530281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:33.618885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f577e2-60ed-4014-bb8e-c28aa8a307a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7a9c1262-2ad4-45bb-8fe5-c1f32c530281 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:33.620237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:33.621763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:33.622699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 221/441] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:06:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:33.623791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f577e2-60ed-4014-bb8e-c28aa8a307a1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.623965 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600b4c-d9db-4a7f-ba61-fcdb03023c2f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.624253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600b4c-d9db-4a7f-ba61-fcdb03023c2f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.624530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600b4c-d9db-4a7f-ba61-fcdb03023c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.624632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f577e2-60ed-4014-bb8e-c28aa8a307a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.625044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f577e2-60ed-4014-bb8e-c28aa8a307a1 None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.630740 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f577e2-60ed-4014-bb8e-c28aa8a307a1 None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:33.632516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f577e2-60ed-4014-bb8e-c28aa8a307a1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:33.634402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 221/442] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:33 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:33.635575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f577e2-60ed-4014-bb8e-c28aa8a307a1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:33.650897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f7600b4c-d9db-4a7f-ba61-fcdb03023c2f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:33.652544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f7600b4c-d9db-4a7f-ba61-fcdb03023c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:33.744498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:33.751927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 222/443] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:33.753260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7600b4c-d9db-4a7f-ba61-fcdb03023c2f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:33.773439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f577e2-60ed-4014-bb8e-c28aa8a307a1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:33.810361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c52f97-d9b6-435a-a414-e91a64a96e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:33.811190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c52f97-d9b6-435a-a414-e91a64a96e5c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.811556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c52f97-d9b6-435a-a414-e91a64a96e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.811833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c52f97-d9b6-435a-a414-e91a64a96e5c None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.812593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c52f97-d9b6-435a-a414-e91a64a96e5c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:33.812903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c52f97-d9b6-435a-a414-e91a64a96e5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:33.814487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c52f97-d9b6-435a-a414-e91a64a96e5c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:33.824522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 222/444] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:33.825427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c52f97-d9b6-435a-a414-e91a64a96e5c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:33.854118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:33.880344 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7600b4c-d9db-4a7f-ba61-fcdb03023c2f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:33.929442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1652b7-d806-4f10-a6e0-1c8e6375c877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:33.930137 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1652b7-d806-4f10-a6e0-1c8e6375c877 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.930432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1652b7-d806-4f10-a6e0-1c8e6375c877 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.930699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1652b7-d806-4f10-a6e0-1c8e6375c877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:33.931528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1652b7-d806-4f10-a6e0-1c8e6375c877 None admin] RBAC: Adding query filter params (group.id=0244ce9c37c24738a2035de27f780da7, role.id=e7180b44fc2945568dfb83dfaf8b5c79, scope.project.id=82df840b0bc44187a1a9db5c2a668847) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:33.931837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1652b7-d806-4f10-a6e0-1c8e6375c877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:33.933286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1652b7-d806-4f10-a6e0-1c8e6375c877 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:33.934108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c52f97-d9b6-435a-a414-e91a64a96e5c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:33.939808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 223/445] 10.222.0.42 () {56 vars in 1376 bytes} [Wed May 25 16:06:33 2022] GET /identity/v3/role_assignments?group.id=0244ce9c37c24738a2035de27f780da7&scope.project.id=82df840b0bc44187a1a9db5c2a668847&role.id=e7180b44fc2945568dfb83dfaf8b5c79 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:33.940619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1652b7-d806-4f10-a6e0-1c8e6375c877 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:33.985889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329a67f-0667-400b-bd72-8a0b86a151fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:33.986724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a67f-0667-400b-bd72-8a0b86a151fa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:33.987010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a67f-0667-400b-bd72-8a0b86a151fa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:33.987415 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a67f-0667-400b-bd72-8a0b86a151fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:34.002025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329a67f-0667-400b-bd72-8a0b86a151fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:34.010039 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a329a67f-0667-400b-bd72-8a0b86a151fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:34.010625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:34.011763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:06:34.012763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:34.015481 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 223/446] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:06:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:34.021785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329a67f-0667-400b-bd72-8a0b86a151fa None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:34.058451 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1652b7-d806-4f10-a6e0-1c8e6375c877 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:34.099062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea920edc-3bbf-4d99-b83e-b3fd7ebc3bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:34.100028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea920edc-3bbf-4d99-b83e-b3fd7ebc3bfe None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:34.100207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea920edc-3bbf-4d99-b83e-b3fd7ebc3bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:34.100540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea920edc-3bbf-4d99-b83e-b3fd7ebc3bfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:34.119321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:34.138727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329a67f-0667-400b-bd72-8a0b86a151fa None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:34.190991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04855b90-5ef1-4e49-8ca7-ad3afe007a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:34.192834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04855b90-5ef1-4e49-8ca7-ad3afe007a31 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:34.192834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04855b90-5ef1-4e49-8ca7-ad3afe007a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:34.192834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04855b90-5ef1-4e49-8ca7-ad3afe007a31 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:34.193072 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04855b90-5ef1-4e49-8ca7-ad3afe007a31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:34.193259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04855b90-5ef1-4e49-8ca7-ad3afe007a31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:34.194762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04855b90-5ef1-4e49-8ca7-ad3afe007a31 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:34.198127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea920edc-3bbf-4d99-b83e-b3fd7ebc3bfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:34.199593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea920edc-3bbf-4d99-b83e-b3fd7ebc3bfe None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:34.205826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 224/447] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:34.210859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04855b90-5ef1-4e49-8ca7-ad3afe007a31 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:34.287694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:34.336680 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 224/448] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:34.343097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea920edc-3bbf-4d99-b83e-b3fd7ebc3bfe None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:34.343761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04855b90-5ef1-4e49-8ca7-ad3afe007a31 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:34.386838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e7b0f-d46a-4a83-846f-3d75c8d03567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:34.387523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e7b0f-d46a-4a83-846f-3d75c8d03567 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:34.388457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e7b0f-d46a-4a83-846f-3d75c8d03567 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:34.388812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e7b0f-d46a-4a83-846f-3d75c8d03567 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:34.412615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:34.479243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e7b0f-d46a-4a83-846f-3d75c8d03567 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:34.480909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e7b0f-d46a-4a83-846f-3d75c8d03567 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:34.481479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea920edc-3bbf-4d99-b83e-b3fd7ebc3bfe None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:34.525002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6cc20-6cc3-4d31-9531-64f66fac89ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:34.526430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6cc20-6cc3-4d31-9531-64f66fac89ef None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:34.526430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6cc20-6cc3-4d31-9531-64f66fac89ef None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:34.526590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6cc20-6cc3-4d31-9531-64f66fac89ef None admin] PATH_INFO: `/v3/projects/affcd8ec317b49faaaa94fdb779a9948` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:34.527659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6cc20-6cc3-4d31-9531-64f66fac89ef None admin] RBAC: Authorizing `identity:get_project(project_id=affcd8ec317b49faaaa94fdb779a9948)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:34.529626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6cc20-6cc3-4d31-9531-64f66fac89ef None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:34.531787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 225/449] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:34 2022] GET /identity/v3/projects/affcd8ec317b49faaaa94fdb779a9948 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:34.537245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6cc20-6cc3-4d31-9531-64f66fac89ef None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:34.568096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:34.596203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 225/450] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 16:06:34.597315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e7b0f-d46a-4a83-846f-3d75c8d03567 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:34.639381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6cc20-6cc3-4d31-9531-64f66fac89ef None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:34.647841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:34.674364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf94eb5-1255-461a-b4cc-246a0ebe0085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:34.675226 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf94eb5-1255-461a-b4cc-246a0ebe0085 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:34.675511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf94eb5-1255-461a-b4cc-246a0ebe0085 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:34.675789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf94eb5-1255-461a-b4cc-246a0ebe0085 None admin] PATH_INFO: `/v3/groups/0244ce9c37c24738a2035de27f780da7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:34.676984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf94eb5-1255-461a-b4cc-246a0ebe0085 None admin] RBAC: Authorizing `identity:get_group(group_id=0244ce9c37c24738a2035de27f780da7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:34.678475 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf94eb5-1255-461a-b4cc-246a0ebe0085 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:34.680104 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 226/451] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:34 2022] GET /identity/v3/groups/0244ce9c37c24738a2035de27f780da7 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:34.685823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf94eb5-1255-461a-b4cc-246a0ebe0085 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:34.699240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e7b0f-d46a-4a83-846f-3d75c8d03567 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:34.734043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4429e-7c80-48e3-bdaa-37a6666bcdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:34.734903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4429e-7c80-48e3-bdaa-37a6666bcdca None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:34.735183 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4429e-7c80-48e3-bdaa-37a6666bcdca None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:34.735465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4429e-7c80-48e3-bdaa-37a6666bcdca None admin] PATH_INFO: `/v3/projects/55f82ef7ffb1469ea7582069b5da4934` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:34.736639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4429e-7c80-48e3-bdaa-37a6666bcdca None admin] RBAC: Authorizing `identity:get_project(project_id=55f82ef7ffb1469ea7582069b5da4934)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:34.738111 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4429e-7c80-48e3-bdaa-37a6666bcdca None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:34.739797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 226/452] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:06:34 2022] GET /identity/v3/projects/55f82ef7ffb1469ea7582069b5da4934 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:34.745364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4429e-7c80-48e3-bdaa-37a6666bcdca None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:34.781207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:34.785501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf94eb5-1255-461a-b4cc-246a0ebe0085 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:34.820583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f459f-9876-4932-9bb9-69f1ed0b192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:34.821289 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f459f-9876-4932-9bb9-69f1ed0b192e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:34.821913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f459f-9876-4932-9bb9-69f1ed0b192e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:34.821913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f459f-9876-4932-9bb9-69f1ed0b192e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:34.822679 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f459f-9876-4932-9bb9-69f1ed0b192e None admin] RBAC: Adding query filter params (group.id=0244ce9c37c24738a2035de27f780da7, role.id=e7180b44fc2945568dfb83dfaf8b5c79, scope.project.id=affcd8ec317b49faaaa94fdb779a9948) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:34.823259 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f459f-9876-4932-9bb9-69f1ed0b192e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:34.824409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f459f-9876-4932-9bb9-69f1ed0b192e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:34.830702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 227/453] 10.222.0.42 () {56 vars in 1376 bytes} [Wed May 25 16:06:34 2022] GET /identity/v3/role_assignments?group.id=0244ce9c37c24738a2035de27f780da7&scope.project.id=affcd8ec317b49faaaa94fdb779a9948&role.id=e7180b44fc2945568dfb83dfaf8b5c79 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:34.849359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4429e-7c80-48e3-bdaa-37a6666bcdca None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:34.869085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:34.889939 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83ac9a-e281-450a-9425-1555ae6627d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:34.890739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83ac9a-e281-450a-9425-1555ae6627d6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:34.891021 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83ac9a-e281-450a-9425-1555ae6627d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:34.891293 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83ac9a-e281-450a-9425-1555ae6627d6 None admin] PATH_INFO: `/v3/groups/0b5721eb211841ccb2b9ede53a4aa1fc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:34.892669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd83ac9a-e281-450a-9425-1555ae6627d6 None admin] RBAC: Authorizing `identity:get_group(group_id=0b5721eb211841ccb2b9ede53a4aa1fc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:34.894230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd83ac9a-e281-450a-9425-1555ae6627d6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:34.895993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 227/454] 10.222.0.42 () {56 vars in 1180 bytes} [Wed May 25 16:06:34 2022] GET /identity/v3/groups/0b5721eb211841ccb2b9ede53a4aa1fc => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:34.902340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f459f-9876-4932-9bb9-69f1ed0b192e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:35.010926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f459f-9876-4932-9bb9-69f1ed0b192e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:35.048356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ac605-11fb-4787-9c4c-069fe7863520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:35.049024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ac605-11fb-4787-9c4c-069fe7863520 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:35.049308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ac605-11fb-4787-9c4c-069fe7863520 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:35.049584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ac605-11fb-4787-9c4c-069fe7863520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:35.050443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ac605-11fb-4787-9c4c-069fe7863520 None admin] RBAC: Adding query filter params (group.id=0b5721eb211841ccb2b9ede53a4aa1fc, role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=55f82ef7ffb1469ea7582069b5da4934) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:35.050759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ac605-11fb-4787-9c4c-069fe7863520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:35.052201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ac605-11fb-4787-9c4c-069fe7863520 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:35.058355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 228/455] 10.222.0.42 () {56 vars in 1376 bytes} [Wed May 25 16:06:34 2022] GET /identity/v3/role_assignments?group.id=0b5721eb211841ccb2b9ede53a4aa1fc&scope.project.id=55f82ef7ffb1469ea7582069b5da4934&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:37.753085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a63102-23d3-478a-8c22-c80aee71f4e4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:37.754055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a63102-23d3-478a-8c22-c80aee71f4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:37.754544 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a63102-23d3-478a-8c22-c80aee71f4e4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:37.756388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 228/456] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:37.761481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c0a1a-029e-474e-99b1-0fd7a6b549b3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:37.762340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c0a1a-029e-474e-99b1-0fd7a6b549b3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:37.763363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c0a1a-029e-474e-99b1-0fd7a6b549b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:37.786718 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cd0c0a1a-029e-474e-99b1-0fd7a6b549b3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:37.788245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cd0c0a1a-029e-474e-99b1-0fd7a6b549b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:37.813502 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:37.874806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 229/457] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:37.903717 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee3603-1c23-4ae0-b7da-f725bd9ff9c3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:37.904515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee3603-1c23-4ae0-b7da-f725bd9ff9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:37.904990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee3603-1c23-4ae0-b7da-f725bd9ff9c3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:37.906671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 229/458] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:37.909746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c0a1a-029e-474e-99b1-0fd7a6b549b3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:37.923339 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e36a3-646f-4573-9c52-6fa38f0574c6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:37.924003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e36a3-646f-4573-9c52-6fa38f0574c6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:37.924496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e36a3-646f-4573-9c52-6fa38f0574c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:37.951153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-962e36a3-646f-4573-9c52-6fa38f0574c6 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:37.952561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-962e36a3-646f-4573-9c52-6fa38f0574c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:38.091785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c0a1a-029e-474e-99b1-0fd7a6b549b3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:38.096230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 230/459] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:06:38.107248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af051d8-0ae5-4bef-b74d-7359fecaa614 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:38.107762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af051d8-0ae5-4bef-b74d-7359fecaa614 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:38.108216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af051d8-0ae5-4bef-b74d-7359fecaa614 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:38.113848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 231/460] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:38.116136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2823109-0166-4a28-a97b-d1d193fd7f42 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:38.116641 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2823109-0166-4a28-a97b-d1d193fd7f42 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:38.117092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2823109-0166-4a28-a97b-d1d193fd7f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:38.134805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c2823109-0166-4a28-a97b-d1d193fd7f42 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:38.136399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c2823109-0166-4a28-a97b-d1d193fd7f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:38.155929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c476d-64cd-46a5-8857-babd52d57440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:38.156996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c476d-64cd-46a5-8857-babd52d57440 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:38.157887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c476d-64cd-46a5-8857-babd52d57440 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:38.158342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c476d-64cd-46a5-8857-babd52d57440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:38.173989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c476d-64cd-46a5-8857-babd52d57440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:38.175567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c476d-64cd-46a5-8857-babd52d57440 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c27c476d-64cd-46a5-8857-babd52d57440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:38.194242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:38.195187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:38.196028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:38.196028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:38.196028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:38.196028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:38.198356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 230/461] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:38.199388 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c476d-64cd-46a5-8857-babd52d57440 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:38.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:38.225737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 232/462] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:06:38.225737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2823109-0166-4a28-a97b-d1d193fd7f42 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:38.275208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:38.338263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c476d-64cd-46a5-8857-babd52d57440 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:38.354039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2823109-0166-4a28-a97b-d1d193fd7f42 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:38.382556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6ee5e-ea61-439b-9939-8ced93891a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:38.383506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6ee5e-ea61-439b-9939-8ced93891a18 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:38.383961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6ee5e-ea61-439b-9939-8ced93891a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:38.384710 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6ee5e-ea61-439b-9939-8ced93891a18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:38.393092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44e4d2-62e6-4227-9452-f0a17bb4e0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:38.394096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44e4d2-62e6-4227-9452-f0a17bb4e0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:38.394648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44e4d2-62e6-4227-9452-f0a17bb4e0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:38.395100 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44e4d2-62e6-4227-9452-f0a17bb4e0f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:38.396006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44e4d2-62e6-4227-9452-f0a17bb4e0f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:38.396511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44e4d2-62e6-4227-9452-f0a17bb4e0f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:38.396613 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6ee5e-ea61-439b-9939-8ced93891a18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:38.398123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6ee5e-ea61-439b-9939-8ced93891a18 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:38.398356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44e4d2-62e6-4227-9452-f0a17bb4e0f9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:38.412052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 233/463] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:38.413295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44e4d2-62e6-4227-9452-f0a17bb4e0f9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8cf6ee5e-ea61-439b-9939-8ced93891a18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:38.420314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:38.421157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:38.422033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:38.422033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:38.422033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:38.422033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:38.423554 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 231/464] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:38.429556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6ee5e-ea61-439b-9939-8ced93891a18 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:38.506211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:38.540149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44e4d2-62e6-4227-9452-f0a17bb4e0f9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:38.542943 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:38.569585 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6ee5e-ea61-439b-9939-8ced93891a18 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:38.594957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b3ab6-2e15-4414-b8b3-e959c72960df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:38.595763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b3ab6-2e15-4414-b8b3-e959c72960df None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:38.596051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b3ab6-2e15-4414-b8b3-e959c72960df None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:38.596332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b3ab6-2e15-4414-b8b3-e959c72960df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:38.611704 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b3ab6-2e15-4414-b8b3-e959c72960df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:38.613120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b3ab6-2e15-4414-b8b3-e959c72960df None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:38.613893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945b93a-1a0c-4b84-86f2-67bb08db079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:38.614829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945b93a-1a0c-4b84-86f2-67bb08db079d None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:38.615451 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945b93a-1a0c-4b84-86f2-67bb08db079d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:38.616022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945b93a-1a0c-4b84-86f2-67bb08db079d None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:38.617098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945b93a-1a0c-4b84-86f2-67bb08db079d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:38.618777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945b93a-1a0c-4b84-86f2-67bb08db079d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9b8b3ab6-2e15-4414-b8b3-e959c72960df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:38.635401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:38.636338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:38.642568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:38.642568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:38.642568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:38.642568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:38.642568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 234/465] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:38.642568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b3ab6-2e15-4414-b8b3-e959c72960df None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:38.652649 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 232/466] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:06:38 2022] POST /identity/v3/users => generated 246 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:06:38.654494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945b93a-1a0c-4b84-86f2-67bb08db079d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:38.817909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945b93a-1a0c-4b84-86f2-67bb08db079d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:38.842810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:38.850858 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b3ab6-2e15-4414-b8b3-e959c72960df None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:38.876133 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37e46c-4583-4b69-a2c1-0f6926cae7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:38.877059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37e46c-4583-4b69-a2c1-0f6926cae7ce None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:38.877347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37e46c-4583-4b69-a2c1-0f6926cae7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:38.877868 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37e46c-4583-4b69-a2c1-0f6926cae7ce None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:38.879197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37e46c-4583-4b69-a2c1-0f6926cae7ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:38.879197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37e46c-4583-4b69-a2c1-0f6926cae7ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:38.880135 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37e46c-4583-4b69-a2c1-0f6926cae7ce None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:38.893295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f870b95-d030-4fc4-90f7-a62d1c47a288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:38.894046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f870b95-d030-4fc4-90f7-a62d1c47a288 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:38.894328 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f870b95-d030-4fc4-90f7-a62d1c47a288 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:38.894602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f870b95-d030-4fc4-90f7-a62d1c47a288 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:38.895327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f870b95-d030-4fc4-90f7-a62d1c47a288 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:38.895632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f870b95-d030-4fc4-90f7-a62d1c47a288 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:38.896035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 233/467] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:38.897293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943b20-bb83-44e5-857a-550288fb6fdc None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:38.897748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943b20-bb83-44e5-857a-550288fb6fdc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:38.897910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f870b95-d030-4fc4-90f7-a62d1c47a288 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:38.897998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943b20-bb83-44e5-857a-550288fb6fdc None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:38.899308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 234/468] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:38.902013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45943b20-bb83-44e5-857a-550288fb6fdc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:38.913428 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 235/469] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:38.914937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f877c-9dcd-432c-8d78-34ebfc46744b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:38.915223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f877c-9dcd-432c-8d78-34ebfc46744b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:38.915496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f877c-9dcd-432c-8d78-34ebfc46744b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:38.932983 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-725f877c-9dcd-432c-8d78-34ebfc46744b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:38.934455 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-725f877c-9dcd-432c-8d78-34ebfc46744b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:38.951277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:39.023020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 236/470] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:39.023971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f877c-9dcd-432c-8d78-34ebfc46744b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:39.034604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45943b20-bb83-44e5-857a-550288fb6fdc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:39.077276 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818f357-81ae-4fe3-ad5b-b9d7f0ece686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:39.077995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818f357-81ae-4fe3-ad5b-b9d7f0ece686 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:39.078277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818f357-81ae-4fe3-ad5b-b9d7f0ece686 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:39.078549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818f357-81ae-4fe3-ad5b-b9d7f0ece686 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:39.079250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818f357-81ae-4fe3-ad5b-b9d7f0ece686 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:39.080695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818f357-81ae-4fe3-ad5b-b9d7f0ece686 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:39.105628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 235/471] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:06:38 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:06:39.106671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818f357-81ae-4fe3-ad5b-b9d7f0ece686 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:39.146617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:39.170626 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f877c-9dcd-432c-8d78-34ebfc46744b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:39.213079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33088a9-4b54-4c4f-9ca2-c3f922e5e17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:39.213801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33088a9-4b54-4c4f-9ca2-c3f922e5e17a None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:39.214089 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33088a9-4b54-4c4f-9ca2-c3f922e5e17a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:39.214364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33088a9-4b54-4c4f-9ca2-c3f922e5e17a None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:39.215146 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33088a9-4b54-4c4f-9ca2-c3f922e5e17a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:39.217368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33088a9-4b54-4c4f-9ca2-c3f922e5e17a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:39.231056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818f357-81ae-4fe3-ad5b-b9d7f0ece686 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:39.242388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 237/472] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:06:39 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:06:39.276800 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9256d-9e62-4e70-b0dd-e768ae74631d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:39.277628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9256d-9e62-4e70-b0dd-e768ae74631d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:39.277943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9256d-9e62-4e70-b0dd-e768ae74631d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:39.278220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9256d-9e62-4e70-b0dd-e768ae74631d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:39.293007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9256d-9e62-4e70-b0dd-e768ae74631d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:39.294669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9256d-9e62-4e70-b0dd-e768ae74631d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-77f9256d-9e62-4e70-b0dd-e768ae74631d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:39.315097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:39.316228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:39.317620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:39.317620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:39.317620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:39.317620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:39.317620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 236/473] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:39.322465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33088a9-4b54-4c4f-9ca2-c3f922e5e17a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:39.328815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:39.475923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33088a9-4b54-4c4f-9ca2-c3f922e5e17a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:39.484030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d9019-f7b9-4e40-8a77-fd85b2a06bf9 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:39.484332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d9019-f7b9-4e40-8a77-fd85b2a06bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:39.484592 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d9019-f7b9-4e40-8a77-fd85b2a06bf9 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:39.487152 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 237/474] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:39.493393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031f5ea-523c-488a-b53d-dcfd73bad9a6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:39.493980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031f5ea-523c-488a-b53d-dcfd73bad9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:39.494476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031f5ea-523c-488a-b53d-dcfd73bad9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:39.513824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5084e7f-8ff6-4b0f-b35a-f44c219372f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:39.514502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5084e7f-8ff6-4b0f-b35a-f44c219372f8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:39.514790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5084e7f-8ff6-4b0f-b35a-f44c219372f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:39.515073 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5084e7f-8ff6-4b0f-b35a-f44c219372f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:39.515741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d031f5ea-523c-488a-b53d-dcfd73bad9a6 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:39.515868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5084e7f-8ff6-4b0f-b35a-f44c219372f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:39.516114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5084e7f-8ff6-4b0f-b35a-f44c219372f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:39.516930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d031f5ea-523c-488a-b53d-dcfd73bad9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:39.517499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5084e7f-8ff6-4b0f-b35a-f44c219372f8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:39.531531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 238/475] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:39.536913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5084e7f-8ff6-4b0f-b35a-f44c219372f8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:39.612912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:39.614779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 238/476] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:39.640396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031f5ea-523c-488a-b53d-dcfd73bad9a6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:39.662702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:39.665818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5084e7f-8ff6-4b0f-b35a-f44c219372f8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:39.712117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d661d-bdbe-4bed-a46e-b830533899c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:39.713884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d661d-bdbe-4bed-a46e-b830533899c7 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:39.715027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d661d-bdbe-4bed-a46e-b830533899c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:39.715642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d661d-bdbe-4bed-a46e-b830533899c7 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:39.718229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d661d-bdbe-4bed-a46e-b830533899c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:39.720044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d661d-bdbe-4bed-a46e-b830533899c7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:39.756586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 239/477] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:06:39 2022] POST /identity/v3/users => generated 249 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:06:39.792891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031f5ea-523c-488a-b53d-dcfd73bad9a6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:39.843907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcac2c-c90d-4d11-af47-4434361320c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:39.845844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcac2c-c90d-4d11-af47-4434361320c1 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:39.845844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcac2c-c90d-4d11-af47-4434361320c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:39.846161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcac2c-c90d-4d11-af47-4434361320c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:39.853740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bcac2c-c90d-4d11-af47-4434361320c1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:39.853740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bcac2c-c90d-4d11-af47-4434361320c1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:39.865751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.sql.core [None req-a1bcac2c-c90d-4d11-af47-4434361320c1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 16:06:39.865751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 16:06:39.865751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [parameters: {'id': 'b4492e999dc440039861e0e01d40eac9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 16:06:39.865751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80167) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a1bcac2c-c90d-4d11-af47-4434361320c1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = self._query(query) May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application conn.query(q) May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result.read() May 25 16:06:39.885232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return ref.to_dict() May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 16:06:39.887097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application next(self.gen) May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application yield resource May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application next(self.gen) May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.session.rollback() May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.force_reraise() May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise self.value May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application session.commit() May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 16:06:39.889954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._prepare_impl() May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.session.flush() May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._flush(objects) May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application compat.raise_( May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application flush_context.execute() May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rec.execute(self) May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 16:06:39.898732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 16:06:39.900781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = self._query(query) May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application conn.query(q) May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result.read() May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 16:06:39.902759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [parameters: {'id': 'b4492e999dc440039861e0e01d40eac9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:39.905578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 16:06:39.907530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:39.914628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 239/478] 10.222.0.42 () {60 vars in 1131 bytes} [Wed May 25 16:06:39 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:39.914791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d661d-bdbe-4bed-a46e-b830533899c7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:40.065976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d661d-bdbe-4bed-a46e-b830533899c7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:40.089173 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:40.111972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fda54-d013-46bd-9a5e-724ba0516a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:40.112790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fda54-d013-46bd-9a5e-724ba0516a11 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:40.113068 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fda54-d013-46bd-9a5e-724ba0516a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:40.113341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fda54-d013-46bd-9a5e-724ba0516a11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:40.124663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27fda54-d013-46bd-9a5e-724ba0516a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f27fda54-d013-46bd-9a5e-724ba0516a11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:40.132922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:40.134885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:40.136492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 240/479] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:06:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:40.141785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcac2c-c90d-4d11-af47-4434361320c1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:40.228524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:40.254478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcac2c-c90d-4d11-af47-4434361320c1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:40.290770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56470c47-bff1-41ea-aa2a-01b1a75cdc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:40.291449 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56470c47-bff1-41ea-aa2a-01b1a75cdc53 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:40.291730 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56470c47-bff1-41ea-aa2a-01b1a75cdc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:40.292001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56470c47-bff1-41ea-aa2a-01b1a75cdc53 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:40.292745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56470c47-bff1-41ea-aa2a-01b1a75cdc53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:40.293046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56470c47-bff1-41ea-aa2a-01b1a75cdc53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:40.295052 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56470c47-bff1-41ea-aa2a-01b1a75cdc53 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:40.308154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 240/480] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:41.417479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955494ed-21a4-4ef7-ac88-6dac5cbd30f3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:41.418202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955494ed-21a4-4ef7-ac88-6dac5cbd30f3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:41.418202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955494ed-21a4-4ef7-ac88-6dac5cbd30f3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:41.420353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 241/481] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:41.426004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a38bb-f2c3-490d-85b1-4f638c294af9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:41.426549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a38bb-f2c3-490d-85b1-4f638c294af9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:41.426549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a38bb-f2c3-490d-85b1-4f638c294af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:41.447329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b62a38bb-f2c3-490d-85b1-4f638c294af9 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:41.448599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b62a38bb-f2c3-490d-85b1-4f638c294af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:41.540080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 241/482] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:41.574653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955494ed-21a4-4ef7-ac88-6dac5cbd30f3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:41.704350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955494ed-21a4-4ef7-ac88-6dac5cbd30f3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:41.741749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0aab49-ca76-4666-9d07-3f4dd2528441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:41.742569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aab49-ca76-4666-9d07-3f4dd2528441 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:41.742868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aab49-ca76-4666-9d07-3f4dd2528441 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:41.743123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aab49-ca76-4666-9d07-3f4dd2528441 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:41.755273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0aab49-ca76-4666-9d07-3f4dd2528441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5a0aab49-ca76-4666-9d07-3f4dd2528441 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:41.763772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:41.765755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:41.766819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 242/483] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:41.773284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a38bb-f2c3-490d-85b1-4f638c294af9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:41.889348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:41.933465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a38bb-f2c3-490d-85b1-4f638c294af9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:41.984212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85efbfd2-3621-4a57-bab5-e584667be99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:41.985231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efbfd2-3621-4a57-bab5-e584667be99e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:41.985728 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efbfd2-3621-4a57-bab5-e584667be99e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:41.986198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efbfd2-3621-4a57-bab5-e584667be99e None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:41.987175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efbfd2-3621-4a57-bab5-e584667be99e None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:41.987726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efbfd2-3621-4a57-bab5-e584667be99e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:41.989567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efbfd2-3621-4a57-bab5-e584667be99e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:42.003310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 242/484] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:42.009094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0aab49-ca76-4666-9d07-3f4dd2528441 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:42.075904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:42.165774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0aab49-ca76-4666-9d07-3f4dd2528441 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:42.218720 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd421c1-e2fa-47ec-92c1-20dfa1fe602f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:42.219525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd421c1-e2fa-47ec-92c1-20dfa1fe602f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.219877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd421c1-e2fa-47ec-92c1-20dfa1fe602f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.220085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd421c1-e2fa-47ec-92c1-20dfa1fe602f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.231029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd421c1-e2fa-47ec-92c1-20dfa1fe602f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:42.233489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd421c1-e2fa-47ec-92c1-20dfa1fe602f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:42.241975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94139e3-b6bf-4c95-b664-c2be705370f3 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.242237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94139e3-b6bf-4c95-b664-c2be705370f3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.242473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94139e3-b6bf-4c95-b664-c2be705370f3 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.244110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 243/485] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:42.251410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bbce4-2881-4a2e-890c-377ec3213acc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.251986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bbce4-2881-4a2e-890c-377ec3213acc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.252458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bbce4-2881-4a2e-890c-377ec3213acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4bd421c1-e2fa-47ec-92c1-20dfa1fe602f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:42.259337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:42.260185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:42.261003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:42.261003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:42.261003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:42.261003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:42.261003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 243/486] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:42.270760 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd421c1-e2fa-47ec-92c1-20dfa1fe602f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:42.282868 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d71bbce4-2881-4a2e-890c-377ec3213acc None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:42.284336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d71bbce4-2881-4a2e-890c-377ec3213acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:42.373978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 244/487] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:42.428612 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:42.433253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71bbce4-2881-4a2e-890c-377ec3213acc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:42.441531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd421c1-e2fa-47ec-92c1-20dfa1fe602f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:42.480070 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4526b18-be64-4dbe-97dd-f2869007b2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:42.480742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4526b18-be64-4dbe-97dd-f2869007b2dc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.481024 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4526b18-be64-4dbe-97dd-f2869007b2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.481490 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4526b18-be64-4dbe-97dd-f2869007b2dc None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.482299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4526b18-be64-4dbe-97dd-f2869007b2dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:42.482609 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4526b18-be64-4dbe-97dd-f2869007b2dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:42.484027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4526b18-be64-4dbe-97dd-f2869007b2dc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:42.498704 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 244/488] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:42.500224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b861bda-3b3f-4356-bf3a-9fac01e5ef37 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.500515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b861bda-3b3f-4356-bf3a-9fac01e5ef37 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.500794 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b861bda-3b3f-4356-bf3a-9fac01e5ef37 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.502645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 245/489] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:42.503612 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b861bda-3b3f-4356-bf3a-9fac01e5ef37 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:42.523115 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:42.578819 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71bbce4-2881-4a2e-890c-377ec3213acc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:42.626081 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350e9d9-0b99-4ffd-8d70-c3b2a809dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:42.626918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350e9d9-0b99-4ffd-8d70-c3b2a809dbfd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.627202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350e9d9-0b99-4ffd-8d70-c3b2a809dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.627477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350e9d9-0b99-4ffd-8d70-c3b2a809dbfd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.635236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b861bda-3b3f-4356-bf3a-9fac01e5ef37 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:42.639775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350e9d9-0b99-4ffd-8d70-c3b2a809dbfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8350e9d9-0b99-4ffd-8d70-c3b2a809dbfd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:42.648321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:42.649244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:42.650100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:42.655117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 245/490] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:42.657630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4e3b-1c42-4b03-8ef1-f9fa1062af48 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.658089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4e3b-1c42-4b03-8ef1-f9fa1062af48 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.658443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4e3b-1c42-4b03-8ef1-f9fa1062af48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.677429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-645a4e3b-1c42-4b03-8ef1-f9fa1062af48 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:42.678792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-645a4e3b-1c42-4b03-8ef1-f9fa1062af48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:42.682414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0feea-6ad2-40f2-812d-9445ea713552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:42.683837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0feea-6ad2-40f2-812d-9445ea713552 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.684122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0feea-6ad2-40f2-812d-9445ea713552 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.684499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0feea-6ad2-40f2-812d-9445ea713552 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.694744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0feea-6ad2-40f2-812d-9445ea713552 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:42.696379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0feea-6ad2-40f2-812d-9445ea713552 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6be0feea-6ad2-40f2-812d-9445ea713552 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:42.713014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:42.714149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:42.715046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:42.715046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:42.715046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:42.715046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:42.715046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:42.715046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:42.715046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:42.715046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:42.715046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 16:06:42.715046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:42.716624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 246/491] 10.222.0.42 () {56 vars in 1093 bytes} [Wed May 25 16:06:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:42.721776 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0feea-6ad2-40f2-812d-9445ea713552 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:42.732430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:42.767638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 246/492] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:42.767638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335596d-be6b-454b-b90c-9fdd299410d8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.767638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335596d-be6b-454b-b90c-9fdd299410d8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.767638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335596d-be6b-454b-b90c-9fdd299410d8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.767638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 247/493] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:42.767638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335596d-be6b-454b-b90c-9fdd299410d8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:42.829065 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:42.835679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0feea-6ad2-40f2-812d-9445ea713552 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:42.870826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c192dd-10e1-47ba-8874-47a0e5982ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:42.871501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c192dd-10e1-47ba-8874-47a0e5982ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.871867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c192dd-10e1-47ba-8874-47a0e5982ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.872419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c192dd-10e1-47ba-8874-47a0e5982ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.873181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c192dd-10e1-47ba-8874-47a0e5982ba1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:42.873491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c192dd-10e1-47ba-8874-47a0e5982ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:42.874929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c192dd-10e1-47ba-8874-47a0e5982ba1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:42.887406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 247/494] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:42.887618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335596d-be6b-454b-b90c-9fdd299410d8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:42.887729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b88ad3-7534-4879-9df6-613961d926e4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.887729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b88ad3-7534-4879-9df6-613961d926e4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.887729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b88ad3-7534-4879-9df6-613961d926e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.901258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-26b88ad3-7534-4879-9df6-613961d926e4 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:42.902500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-26b88ad3-7534-4879-9df6-613961d926e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:42.925655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b04b8-4dec-4e64-91e3-47b91466ef39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:42.926363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b04b8-4dec-4e64-91e3-47b91466ef39 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:42.926646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b04b8-4dec-4e64-91e3-47b91466ef39 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:42.926925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b04b8-4dec-4e64-91e3-47b91466ef39 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:42.927762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b04b8-4dec-4e64-91e3-47b91466ef39 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:42.928070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b04b8-4dec-4e64-91e3-47b91466ef39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:42.929504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b04b8-4dec-4e64-91e3-47b91466ef39 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:42.969369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 248/495] 10.222.0.42 () {56 vars in 1133 bytes} [Wed May 25 16:06:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:42.970068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b04b8-4dec-4e64-91e3-47b91466ef39 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:42.993041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 248/496] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:42.993041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b88ad3-7534-4879-9df6-613961d926e4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:42.993041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:43.100076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b04b8-4dec-4e64-91e3-47b91466ef39 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:43.109537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b88ad3-7534-4879-9df6-613961d926e4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:43.112659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:43.139327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bf746-d34d-425f-ba68-eed17f285352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:43.140139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bf746-d34d-425f-ba68-eed17f285352 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:43.140428 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bf746-d34d-425f-ba68-eed17f285352 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:43.140711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bf746-d34d-425f-ba68-eed17f285352 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:43.148448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e7464-5c10-4ade-a30b-954c53eb7f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:43.149311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e7464-5c10-4ade-a30b-954c53eb7f5d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:43.149602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e7464-5c10-4ade-a30b-954c53eb7f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:43.149913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e7464-5c10-4ade-a30b-954c53eb7f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:43.152048 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bf746-d34d-425f-ba68-eed17f285352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-446bf746-d34d-425f-ba68-eed17f285352 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:43.160420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:43.161405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 249/497] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:43.162644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bf746-d34d-425f-ba68-eed17f285352 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:43.168302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e7464-5c10-4ade-a30b-954c53eb7f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:43.170610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e7464-5c10-4ade-a30b-954c53eb7f5d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3b5e7464-5c10-4ade-a30b-954c53eb7f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:43.190440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:43.191538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:43.192408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:43.192408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:43.192408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:43.192408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:43.192408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 249/498] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:43.193200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e7464-5c10-4ade-a30b-954c53eb7f5d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:43.269334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:43.309817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bf746-d34d-425f-ba68-eed17f285352 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:43.333040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e7464-5c10-4ade-a30b-954c53eb7f5d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:43.348252 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23095b-ca00-402b-92c2-7933af5a7bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:43.348959 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23095b-ca00-402b-92c2-7933af5a7bcf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:43.349232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23095b-ca00-402b-92c2-7933af5a7bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:43.349513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23095b-ca00-402b-92c2-7933af5a7bcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:43.375502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26211bf-4992-4c3f-b359-d39107a18812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:43.376209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26211bf-4992-4c3f-b359-d39107a18812 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:43.376488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26211bf-4992-4c3f-b359-d39107a18812 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:43.376759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26211bf-4992-4c3f-b359-d39107a18812 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:43.387158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26211bf-4992-4c3f-b359-d39107a18812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:43.394775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d26211bf-4992-4c3f-b359-d39107a18812 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:43.395950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:43.397029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 250/499] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:43.398151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26211bf-4992-4c3f-b359-d39107a18812 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:43.428569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23095b-ca00-402b-92c2-7933af5a7bcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:43.430104 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23095b-ca00-402b-92c2-7933af5a7bcf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:43.504680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26211bf-4992-4c3f-b359-d39107a18812 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:43.519015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:43.531790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 250/500] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:43.532636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23095b-ca00-402b-92c2-7933af5a7bcf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:43.544584 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc6167-5208-4233-946d-baaf1d63016c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:43.545359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6167-5208-4233-946d-baaf1d63016c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:43.545641 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6167-5208-4233-946d-baaf1d63016c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:43.545944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6167-5208-4233-946d-baaf1d63016c None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:43.546692 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc6167-5208-4233-946d-baaf1d63016c None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:43.546998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc6167-5208-4233-946d-baaf1d63016c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:43.548376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc6167-5208-4233-946d-baaf1d63016c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:43.556642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 251/501] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:43.557979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a775eaa-2940-4f97-bc4a-98d14f043244 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:43.558270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a775eaa-2940-4f97-bc4a-98d14f043244 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:43.558552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a775eaa-2940-4f97-bc4a-98d14f043244 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:43.560401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 252/502] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:43.561344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a775eaa-2940-4f97-bc4a-98d14f043244 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:43.611192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:43.643452 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23095b-ca00-402b-92c2-7933af5a7bcf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:43.668081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a775eaa-2940-4f97-bc4a-98d14f043244 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:43.676701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1631e-a0c2-4541-af0f-ee1a3a751897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:43.677367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1631e-a0c2-4541-af0f-ee1a3a751897 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:43.677649 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1631e-a0c2-4541-af0f-ee1a3a751897 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:43.677963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1631e-a0c2-4541-af0f-ee1a3a751897 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:43.678681 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1631e-a0c2-4541-af0f-ee1a3a751897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:43.678985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1631e-a0c2-4541-af0f-ee1a3a751897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:43.680353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1631e-a0c2-4541-af0f-ee1a3a751897 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:43.692135 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 251/503] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:43.692944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1631e-a0c2-4541-af0f-ee1a3a751897 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:43.702826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe86950-8ca9-4878-bff7-b0de5c25ab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:43.703497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86950-8ca9-4878-bff7-b0de5c25ab82 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:43.703783 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86950-8ca9-4878-bff7-b0de5c25ab82 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:43.704060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86950-8ca9-4878-bff7-b0de5c25ab82 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:43.704925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe86950-8ca9-4878-bff7-b0de5c25ab82 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:43.705118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe86950-8ca9-4878-bff7-b0de5c25ab82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:43.706517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe86950-8ca9-4878-bff7-b0de5c25ab82 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:43.715577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 253/504] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:43.716409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe86950-8ca9-4878-bff7-b0de5c25ab82 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:43.760626 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:43.807058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1631e-a0c2-4541-af0f-ee1a3a751897 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:43.821795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe86950-8ca9-4878-bff7-b0de5c25ab82 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:43.835981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:43.844158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32465aac-4e2e-4c91-baa8-0eabec9af7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:43.844956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32465aac-4e2e-4c91-baa8-0eabec9af7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:43.845237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32465aac-4e2e-4c91-baa8-0eabec9af7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:43.845513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32465aac-4e2e-4c91-baa8-0eabec9af7b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:43.856859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa67b6-f684-4b7f-a4ca-22961a561b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:43.857019 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32465aac-4e2e-4c91-baa8-0eabec9af7b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:43.857662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa67b6-f684-4b7f-a4ca-22961a561b86 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:43.857977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa67b6-f684-4b7f-a4ca-22961a561b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:43.858249 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa67b6-f684-4b7f-a4ca-22961a561b86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:43.858454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32465aac-4e2e-4c91-baa8-0eabec9af7b0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:43.868683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa67b6-f684-4b7f-a4ca-22961a561b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:43.870143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa67b6-f684-4b7f-a4ca-22961a561b86 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-32465aac-4e2e-4c91-baa8-0eabec9af7b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:43.875570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:43.876399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:43.877250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:43.877250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:43.877250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:43.877250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:43.877250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 252/505] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:43.877999 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e06a6-240b-4f68-8e7c-cbe946bb981a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:43.878246 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e06a6-240b-4f68-8e7c-cbe946bb981a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:43.878533 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e06a6-240b-4f68-8e7c-cbe946bb981a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0baa67b6-f684-4b7f-a4ca-22961a561b86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:43.891372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:43.892587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:43.893430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:43.893430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:43.893430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:43.893430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:43.893430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 254/506] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:43.893430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa67b6-f684-4b7f-a4ca-22961a561b86 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:43.895471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4b8e06a6-240b-4f68-8e7c-cbe946bb981a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:43.896581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b8e06a6-240b-4f68-8e7c-cbe946bb981a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:43.963049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 253/507] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:43.964146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e06a6-240b-4f68-8e7c-cbe946bb981a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:43.988140 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:43.998658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa67b6-f684-4b7f-a4ca-22961a561b86 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.032447 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918f9a1-74c4-47cf-9627-46c558fba240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.033250 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918f9a1-74c4-47cf-9627-46c558fba240 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.033529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918f9a1-74c4-47cf-9627-46c558fba240 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.034091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918f9a1-74c4-47cf-9627-46c558fba240 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.041691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918f9a1-74c4-47cf-9627-46c558fba240 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.044241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918f9a1-74c4-47cf-9627-46c558fba240 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c918f9a1-74c4-47cf-9627-46c558fba240 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:44.058806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:44.061199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 255/508] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:44.062504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918f9a1-74c4-47cf-9627-46c558fba240 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:44.070000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e06a6-240b-4f68-8e7c-cbe946bb981a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.084121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:44.105373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69321f32-a143-4a16-a2ae-3cc4fa4ab2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.106795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69321f32-a143-4a16-a2ae-3cc4fa4ab2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.107083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69321f32-a143-4a16-a2ae-3cc4fa4ab2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.107355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69321f32-a143-4a16-a2ae-3cc4fa4ab2e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.117202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69321f32-a143-4a16-a2ae-3cc4fa4ab2e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.118684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69321f32-a143-4a16-a2ae-3cc4fa4ab2e0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-69321f32-a143-4a16-a2ae-3cc4fa4ab2e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:44.137020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:44.138369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:44.139330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:44.139330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:44.139330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:44.139330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:44.139330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 254/509] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:44.139330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69321f32-a143-4a16-a2ae-3cc4fa4ab2e0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:44.171975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918f9a1-74c4-47cf-9627-46c558fba240 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.209455 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358fa6b-2a1b-4e51-b880-5ff794f95013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.210238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358fa6b-2a1b-4e51-b880-5ff794f95013 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.210535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358fa6b-2a1b-4e51-b880-5ff794f95013 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.210819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358fa6b-2a1b-4e51-b880-5ff794f95013 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.211547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358fa6b-2a1b-4e51-b880-5ff794f95013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:44.211861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358fa6b-2a1b-4e51-b880-5ff794f95013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.213247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358fa6b-2a1b-4e51-b880-5ff794f95013 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.225263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 256/510] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:44.226121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358fa6b-2a1b-4e51-b880-5ff794f95013 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:44.239133 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:44.243394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69321f32-a143-4a16-a2ae-3cc4fa4ab2e0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.276986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314528f-8ac5-4a25-94ac-d573037e8002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.277654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314528f-8ac5-4a25-94ac-d573037e8002 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.277966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314528f-8ac5-4a25-94ac-d573037e8002 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.278759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314528f-8ac5-4a25-94ac-d573037e8002 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.279483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314528f-8ac5-4a25-94ac-d573037e8002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:44.279788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314528f-8ac5-4a25-94ac-d573037e8002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.281160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314528f-8ac5-4a25-94ac-d573037e8002 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.292806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 255/511] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:44.293611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314528f-8ac5-4a25-94ac-d573037e8002 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:44.319421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:44.331587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358fa6b-2a1b-4e51-b880-5ff794f95013 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.367805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ed3da-7fca-4737-bfd7-ff0f70823f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.368593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ed3da-7fca-4737-bfd7-ff0f70823f0c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.368878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ed3da-7fca-4737-bfd7-ff0f70823f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.369156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ed3da-7fca-4737-bfd7-ff0f70823f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.380128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ed3da-7fca-4737-bfd7-ff0f70823f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.381590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ed3da-7fca-4737-bfd7-ff0f70823f0c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.397896 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314528f-8ac5-4a25-94ac-d573037e8002 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a79ed3da-7fca-4737-bfd7-ff0f70823f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:44.399611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:44.400630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:44.401526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:44.401526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:44.401526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:44.401526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:44.401526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 257/512] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:44.401936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ed3da-7fca-4737-bfd7-ff0f70823f0c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:44.439712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46345df2-9365-4156-ab80-9bf383c769ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.440395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46345df2-9365-4156-ab80-9bf383c769ee None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.440678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46345df2-9365-4156-ab80-9bf383c769ee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.440962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46345df2-9365-4156-ab80-9bf383c769ee None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.441824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46345df2-9365-4156-ab80-9bf383c769ee None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:44.442138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46345df2-9365-4156-ab80-9bf383c769ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.443569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46345df2-9365-4156-ab80-9bf383c769ee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.460809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 256/513] 10.222.0.42 () {56 vars in 1137 bytes} [Wed May 25 16:06:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:44.461635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46345df2-9365-4156-ab80-9bf383c769ee None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:44.474706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:44.514140 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ed3da-7fca-4737-bfd7-ff0f70823f0c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.544544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:44.549394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbb535-3aef-41a1-bc27-f370b46b9ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.550129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbb535-3aef-41a1-bc27-f370b46b9ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.550416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbb535-3aef-41a1-bc27-f370b46b9ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.550688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbb535-3aef-41a1-bc27-f370b46b9ef6 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.551416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbb535-3aef-41a1-bc27-f370b46b9ef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:44.551719 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbb535-3aef-41a1-bc27-f370b46b9ef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.553083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbb535-3aef-41a1-bc27-f370b46b9ef6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.563125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46345df2-9365-4156-ab80-9bf383c769ee None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.566854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 258/514] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:44.567689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbb535-3aef-41a1-bc27-f370b46b9ef6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:44.603950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682e3c2-9bfd-4d81-a1cd-f523a710db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.604739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682e3c2-9bfd-4d81-a1cd-f523a710db41 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.605024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682e3c2-9bfd-4d81-a1cd-f523a710db41 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.605299 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682e3c2-9bfd-4d81-a1cd-f523a710db41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.613049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682e3c2-9bfd-4d81-a1cd-f523a710db41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.614846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682e3c2-9bfd-4d81-a1cd-f523a710db41 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6682e3c2-9bfd-4d81-a1cd-f523a710db41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:06:44.629064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:44.630131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:44.631037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:06:44.631037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:06:44.631037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:06:44.631037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:06:44.631037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:06:44.631037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:06:44.631037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:44.631037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:44.634994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 257/515] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:44.635671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682e3c2-9bfd-4d81-a1cd-f523a710db41 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:44.670596 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbb535-3aef-41a1-bc27-f370b46b9ef6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.692608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:44.705625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ecaf0-cf0d-44c5-8815-acc12f795016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.706449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ecaf0-cf0d-44c5-8815-acc12f795016 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.706726 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ecaf0-cf0d-44c5-8815-acc12f795016 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.706998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ecaf0-cf0d-44c5-8815-acc12f795016 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.714603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ecaf0-cf0d-44c5-8815-acc12f795016 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.716913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ecaf0-cf0d-44c5-8815-acc12f795016 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d66ecaf0-cf0d-44c5-8815-acc12f795016 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:44.731466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:44.732456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:44.733343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:44.733343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:44.733343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:44.733343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:44.733343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:44.733343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:44.733343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:44.733343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:44.733343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 16:06:44.733343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:44.733343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 259/516] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:44.733963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ecaf0-cf0d-44c5-8815-acc12f795016 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:44.743158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682e3c2-9bfd-4d81-a1cd-f523a710db41 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.779271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e15258-0806-4915-b579-a7fc000f6aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.779934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e15258-0806-4915-b579-a7fc000f6aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.780220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e15258-0806-4915-b579-a7fc000f6aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.780497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e15258-0806-4915-b579-a7fc000f6aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.781228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e15258-0806-4915-b579-a7fc000f6aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:44.781527 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e15258-0806-4915-b579-a7fc000f6aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.782951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e15258-0806-4915-b579-a7fc000f6aa2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.795618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 258/517] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:44.796450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e15258-0806-4915-b579-a7fc000f6aa2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:44.802296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:44.848049 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ecaf0-cf0d-44c5-8815-acc12f795016 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.882101 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91190568-0083-4b61-a39a-e02c7812efef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.882749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91190568-0083-4b61-a39a-e02c7812efef None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.883033 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91190568-0083-4b61-a39a-e02c7812efef None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.883310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91190568-0083-4b61-a39a-e02c7812efef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.899718 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e15258-0806-4915-b579-a7fc000f6aa2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:44.933208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:44.936147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dddce-0cf6-4422-b8ff-427658dabec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:44.936957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dddce-0cf6-4422-b8ff-427658dabec1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:44.937238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dddce-0cf6-4422-b8ff-427658dabec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:44.937514 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dddce-0cf6-4422-b8ff-427658dabec1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:44.944772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dddce-0cf6-4422-b8ff-427658dabec1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.946193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dddce-0cf6-4422-b8ff-427658dabec1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.952023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91190568-0083-4b61-a39a-e02c7812efef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:44.953479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91190568-0083-4b61-a39a-e02c7812efef None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-617dddce-0cf6-4422-b8ff-427658dabec1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:44.962498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:44.963422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:44.964325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:44.964325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:44.964325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:44.964325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:44.964325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:44.964325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:44.964325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:44.964325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:44.964325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 16:06:44.964325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:44.964325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 259/518] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:44.964879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dddce-0cf6-4422-b8ff-427658dabec1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:45.027100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:45.056423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 260/519] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:45.057282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91190568-0083-4b61-a39a-e02c7812efef None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:45.066646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dddce-0cf6-4422-b8ff-427658dabec1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:45.102788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44211c-45ad-4a43-b2a2-e3ed4aea1ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:45.103478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44211c-45ad-4a43-b2a2-e3ed4aea1ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:45.103767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44211c-45ad-4a43-b2a2-e3ed4aea1ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:45.104043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44211c-45ad-4a43-b2a2-e3ed4aea1ab7 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:45.104844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44211c-45ad-4a43-b2a2-e3ed4aea1ab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:45.105153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44211c-45ad-4a43-b2a2-e3ed4aea1ab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:45.107017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44211c-45ad-4a43-b2a2-e3ed4aea1ab7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:45.126586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 260/520] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:06:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:45.127414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44211c-45ad-4a43-b2a2-e3ed4aea1ab7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:45.143477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:45.162300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91190568-0083-4b61-a39a-e02c7812efef None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:45.196577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffd52c-36f2-4eb6-a19c-78d706faf4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:45.197176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffd52c-36f2-4eb6-a19c-78d706faf4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:45.197454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffd52c-36f2-4eb6-a19c-78d706faf4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:45.197747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffd52c-36f2-4eb6-a19c-78d706faf4d1 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:45.198590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffd52c-36f2-4eb6-a19c-78d706faf4d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:45.198899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffd52c-36f2-4eb6-a19c-78d706faf4d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:45.200939 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffd52c-36f2-4eb6-a19c-78d706faf4d1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:45.229941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44211c-45ad-4a43-b2a2-e3ed4aea1ab7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:45.236537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 261/521] 10.222.0.42 () {56 vars in 1137 bytes} [Wed May 25 16:06:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:45.237378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffd52c-36f2-4eb6-a19c-78d706faf4d1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:45.254060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:45.266930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e4c17-793d-41b2-bad5-2df3d4720db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:45.267602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e4c17-793d-41b2-bad5-2df3d4720db0 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:45.267885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e4c17-793d-41b2-bad5-2df3d4720db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:45.268156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e4c17-793d-41b2-bad5-2df3d4720db0 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:45.268856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e4c17-793d-41b2-bad5-2df3d4720db0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:45.270323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e4c17-793d-41b2-bad5-2df3d4720db0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:45.290580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 261/522] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:06:45 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:06:45.291406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e4c17-793d-41b2-bad5-2df3d4720db0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:45.349688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffd52c-36f2-4eb6-a19c-78d706faf4d1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:45.392700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669e468-cc25-4248-ae44-3d1c771dc976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:45.393828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669e468-cc25-4248-ae44-3d1c771dc976 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:45.394300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669e468-cc25-4248-ae44-3d1c771dc976 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:45.394678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669e468-cc25-4248-ae44-3d1c771dc976 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:45.395567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669e468-cc25-4248-ae44-3d1c771dc976 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:45.395878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669e468-cc25-4248-ae44-3d1c771dc976 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:45.397372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669e468-cc25-4248-ae44-3d1c771dc976 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:45.409615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:45.417539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e4c17-793d-41b2-bad5-2df3d4720db0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:45.427393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 262/523] 10.222.0.42 () {56 vars in 1139 bytes} [Wed May 25 16:06:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:45.427393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669e468-cc25-4248-ae44-3d1c771dc976 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:45.459923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2613007-37e4-40e8-a6a8-622ded31f803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:45.460648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2613007-37e4-40e8-a6a8-622ded31f803 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:45.460943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2613007-37e4-40e8-a6a8-622ded31f803 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:45.461200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2613007-37e4-40e8-a6a8-622ded31f803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:45.474485 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2613007-37e4-40e8-a6a8-622ded31f803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:45.475972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2613007-37e4-40e8-a6a8-622ded31f803 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f2613007-37e4-40e8-a6a8-622ded31f803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:45.496595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:45.497523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:45.498778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:45.498778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:45.498778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:45.498778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:45.498778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 262/524] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:45.499043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2613007-37e4-40e8-a6a8-622ded31f803 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:45.535643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:45.549005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669e468-cc25-4248-ae44-3d1c771dc976 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:45.587806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72965d09-fb4f-4340-9bdb-0e2ee3f4dba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:45.588465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72965d09-fb4f-4340-9bdb-0e2ee3f4dba2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:45.588751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72965d09-fb4f-4340-9bdb-0e2ee3f4dba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:45.589024 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72965d09-fb4f-4340-9bdb-0e2ee3f4dba2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:45.589925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72965d09-fb4f-4340-9bdb-0e2ee3f4dba2 None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=6e6795b5b5cf4499ad494b875756ba71) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:45.590231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72965d09-fb4f-4340-9bdb-0e2ee3f4dba2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:45.591842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72965d09-fb4f-4340-9bdb-0e2ee3f4dba2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:45.601771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 263/525] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:45 2022] GET /identity/v3/role_assignments?user.id=6e6795b5b5cf4499ad494b875756ba71&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:45.601771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72965d09-fb4f-4340-9bdb-0e2ee3f4dba2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:45.622788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2613007-37e4-40e8-a6a8-622ded31f803 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:45.665054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8027bd-3ab2-4aa5-8d28-ffd204e40ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:45.666179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8027bd-3ab2-4aa5-8d28-ffd204e40ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:45.666798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8027bd-3ab2-4aa5-8d28-ffd204e40ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:45.667313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8027bd-3ab2-4aa5-8d28-ffd204e40ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:45.671153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:45.716044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72965d09-fb4f-4340-9bdb-0e2ee3f4dba2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:45.755434 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8027bd-3ab2-4aa5-8d28-ffd204e40ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:45.756483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0a29b-afcc-47b4-a4ea-afc9b048a350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:45.757152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0a29b-afcc-47b4-a4ea-afc9b048a350 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:45.757435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0a29b-afcc-47b4-a4ea-afc9b048a350 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:45.757726 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0a29b-afcc-47b4-a4ea-afc9b048a350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:45.758214 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8027bd-3ab2-4aa5-8d28-ffd204e40ec5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:45.794695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:45.837693 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0a29b-afcc-47b4-a4ea-afc9b048a350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:45.839966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0a29b-afcc-47b4-a4ea-afc9b048a350 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:45.883921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 263/526] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:45.885253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8027bd-3ab2-4aa5-8d28-ffd204e40ec5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:45.924532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:45.973176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 264/527] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:45.974577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0a29b-afcc-47b4-a4ea-afc9b048a350 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:46.020674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8027bd-3ab2-4aa5-8d28-ffd204e40ec5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:46.061036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:46.068554 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3bccf-84f3-4643-96c5-db0b22fa6a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:46.069234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3bccf-84f3-4643-96c5-db0b22fa6a8f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:46.069517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3bccf-84f3-4643-96c5-db0b22fa6a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:46.069819 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3bccf-84f3-4643-96c5-db0b22fa6a8f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:46.070870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3bccf-84f3-4643-96c5-db0b22fa6a8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:46.071184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3bccf-84f3-4643-96c5-db0b22fa6a8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:46.073206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3bccf-84f3-4643-96c5-db0b22fa6a8f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:46.089333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 264/528] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:46.091326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3bccf-84f3-4643-96c5-db0b22fa6a8f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:46.094449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0a29b-afcc-47b4-a4ea-afc9b048a350 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:46.157561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df935c-ec9d-4ce0-a223-a5120a78ff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:46.158647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df935c-ec9d-4ce0-a223-a5120a78ff39 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:46.159184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df935c-ec9d-4ce0-a223-a5120a78ff39 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:46.159459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df935c-ec9d-4ce0-a223-a5120a78ff39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:46.177263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df935c-ec9d-4ce0-a223-a5120a78ff39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:46.178786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df935c-ec9d-4ce0-a223-a5120a78ff39 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f4df935c-ec9d-4ce0-a223-a5120a78ff39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:46.203472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:46.204421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:46.205280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:46.205280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:46.205280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:46.205280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:46.209730 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 265/529] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:46.210768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df935c-ec9d-4ce0-a223-a5120a78ff39 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:46.224081 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3bccf-84f3-4643-96c5-db0b22fa6a8f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:46.232243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:46.263745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095ebe9-ec26-4b18-8fbe-66afedd3fe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:46.264945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095ebe9-ec26-4b18-8fbe-66afedd3fe52 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:46.265568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095ebe9-ec26-4b18-8fbe-66afedd3fe52 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:46.266070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095ebe9-ec26-4b18-8fbe-66afedd3fe52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:46.278294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095ebe9-ec26-4b18-8fbe-66afedd3fe52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:46.280084 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095ebe9-ec26-4b18-8fbe-66afedd3fe52 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8095ebe9-ec26-4b18-8fbe-66afedd3fe52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:46.301569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:46.302508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:46.303363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:46.303363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:46.303363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:46.303363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:46.310294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 265/530] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:46.311429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095ebe9-ec26-4b18-8fbe-66afedd3fe52 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:46.340096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df935c-ec9d-4ce0-a223-a5120a78ff39 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:46.360799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:46.391363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b586f-811e-443a-b943-df4cafb11029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:46.392171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b586f-811e-443a-b943-df4cafb11029 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:46.392454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b586f-811e-443a-b943-df4cafb11029 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:46.392728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b586f-811e-443a-b943-df4cafb11029 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:46.401436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b586f-811e-443a-b943-df4cafb11029 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:46.403535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b586f-811e-443a-b943-df4cafb11029 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4b7b586f-811e-443a-b943-df4cafb11029 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:06:46.427384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:46.428521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:46.429377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:06:46.429377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:06:46.429377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:06:46.429377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:06:46.429377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:06:46.429377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:06:46.429377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:46.429377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:46.431098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 266/531] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:46.432228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b586f-811e-443a-b943-df4cafb11029 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:46.436880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095ebe9-ec26-4b18-8fbe-66afedd3fe52 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:46.474599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cb843-8c2a-4b68-9937-a5e8eb4a0936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:46.475615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cb843-8c2a-4b68-9937-a5e8eb4a0936 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:46.476103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cb843-8c2a-4b68-9937-a5e8eb4a0936 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:46.476582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cb843-8c2a-4b68-9937-a5e8eb4a0936 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:46.477555 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836cb843-8c2a-4b68-9937-a5e8eb4a0936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:46.478093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836cb843-8c2a-4b68-9937-a5e8eb4a0936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:46.479837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836cb843-8c2a-4b68-9937-a5e8eb4a0936 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:46.504672 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 266/532] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:46.505971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cb843-8c2a-4b68-9937-a5e8eb4a0936 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:46.536099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:46.564021 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b586f-811e-443a-b943-df4cafb11029 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:46.600872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e81deb-7058-4efc-b841-8d22cb214f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:46.601821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e81deb-7058-4efc-b841-8d22cb214f8e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:46.602335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e81deb-7058-4efc-b841-8d22cb214f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:46.602797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e81deb-7058-4efc-b841-8d22cb214f8e None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:46.603756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e81deb-7058-4efc-b841-8d22cb214f8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:46.604269 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e81deb-7058-4efc-b841-8d22cb214f8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:46.605914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e81deb-7058-4efc-b841-8d22cb214f8e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:46.619559 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 267/533] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:46.621154 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e81deb-7058-4efc-b841-8d22cb214f8e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:46.632551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cb843-8c2a-4b68-9937-a5e8eb4a0936 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:46.634269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:46.673584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d00fcc-34e6-49c2-9215-a4069ae676e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:46.674652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d00fcc-34e6-49c2-9215-a4069ae676e6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:46.674883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d00fcc-34e6-49c2-9215-a4069ae676e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:46.675162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d00fcc-34e6-49c2-9215-a4069ae676e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:46.675952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d00fcc-34e6-49c2-9215-a4069ae676e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:46.676314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d00fcc-34e6-49c2-9215-a4069ae676e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:46.677952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d00fcc-34e6-49c2-9215-a4069ae676e6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:46.692109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 267/534] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:46.692898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d00fcc-34e6-49c2-9215-a4069ae676e6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:46.734305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e81deb-7058-4efc-b841-8d22cb214f8e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:46.770945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f5e54-f54b-4b3f-b5f0-fa5bd91a8946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:46.772088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f5e54-f54b-4b3f-b5f0-fa5bd91a8946 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:46.772602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f5e54-f54b-4b3f-b5f0-fa5bd91a8946 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:46.773197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f5e54-f54b-4b3f-b5f0-fa5bd91a8946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:46.775532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:46.781800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f5e54-f54b-4b3f-b5f0-fa5bd91a8946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:46.784000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f5e54-f54b-4b3f-b5f0-fa5bd91a8946 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-595f5e54-f54b-4b3f-b5f0-fa5bd91a8946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:06:46.801748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:46.802678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:46.803531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:06:46.803531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:06:46.803531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:06:46.803531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:06:46.803531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:06:46.803531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:06:46.803531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:46.803531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:46.805833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 268/535] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:46.807011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f5e54-f54b-4b3f-b5f0-fa5bd91a8946 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:46.816310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d00fcc-34e6-49c2-9215-a4069ae676e6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:46.867298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de52b43-66f4-4bff-a690-a7e3a370ac42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:46.868580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de52b43-66f4-4bff-a690-a7e3a370ac42 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:46.869072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de52b43-66f4-4bff-a690-a7e3a370ac42 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:46.869577 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de52b43-66f4-4bff-a690-a7e3a370ac42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:46.878180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de52b43-66f4-4bff-a690-a7e3a370ac42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:46.880013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de52b43-66f4-4bff-a690-a7e3a370ac42 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-2de52b43-66f4-4bff-a690-a7e3a370ac42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:06:46.895834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:46.896733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:46.897569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:06:46.897569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:06:46.897569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:06:46.897569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:06:46.897569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:06:46.897569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:06:46.897569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:46.897569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:46.900118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 268/536] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:46.901116 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de52b43-66f4-4bff-a690-a7e3a370ac42 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:46.928596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:46.936892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f5e54-f54b-4b3f-b5f0-fa5bd91a8946 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:46.975606 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bc0c1-cadc-423d-9b47-d8813fe2f1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:46.976282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bc0c1-cadc-423d-9b47-d8813fe2f1bd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:46.976570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bc0c1-cadc-423d-9b47-d8813fe2f1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:46.976848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bc0c1-cadc-423d-9b47-d8813fe2f1bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:46.977694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bc0c1-cadc-423d-9b47-d8813fe2f1bd None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=b9913b9eb502434985f84c4a11632ccc) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:46.978116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bc0c1-cadc-423d-9b47-d8813fe2f1bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:46.979606 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bc0c1-cadc-423d-9b47-d8813fe2f1bd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:46.987109 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 269/537] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:46 2022] GET /identity/v3/role_assignments?user.id=b9913b9eb502434985f84c4a11632ccc&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:46.987999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bc0c1-cadc-423d-9b47-d8813fe2f1bd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:47.014989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de52b43-66f4-4bff-a690-a7e3a370ac42 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:47.053986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea189be-04b1-4f77-a200-dc5895fbd002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:47.055422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea189be-04b1-4f77-a200-dc5895fbd002 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:47.055916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea189be-04b1-4f77-a200-dc5895fbd002 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:47.056464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea189be-04b1-4f77-a200-dc5895fbd002 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:47.057546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea189be-04b1-4f77-a200-dc5895fbd002 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:47.057760 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:47.058532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea189be-04b1-4f77-a200-dc5895fbd002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:47.060022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea189be-04b1-4f77-a200-dc5895fbd002 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:47.079614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 269/538] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:47.081101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea189be-04b1-4f77-a200-dc5895fbd002 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:47.119250 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bc0c1-cadc-423d-9b47-d8813fe2f1bd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:47.188598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:47.190790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1c4c1-d33f-4cef-a0b1-5fc5840c81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:47.191807 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1c4c1-d33f-4cef-a0b1-5fc5840c81de None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:47.192399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1c4c1-d33f-4cef-a0b1-5fc5840c81de None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:47.192861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1c4c1-d33f-4cef-a0b1-5fc5840c81de None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:47.193892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1c4c1-d33f-4cef-a0b1-5fc5840c81de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:47.194944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1c4c1-d33f-4cef-a0b1-5fc5840c81de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:47.197370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1c4c1-d33f-4cef-a0b1-5fc5840c81de None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:47.205746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea189be-04b1-4f77-a200-dc5895fbd002 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:47.234633 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 270/539] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:47.242825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1c4c1-d33f-4cef-a0b1-5fc5840c81de None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:47.242986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02023d75-2cad-479e-9a9b-345e27b47fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:47.243564 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02023d75-2cad-479e-9a9b-345e27b47fcc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:47.243959 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02023d75-2cad-479e-9a9b-345e27b47fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:47.244295 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02023d75-2cad-479e-9a9b-345e27b47fcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:47.245198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02023d75-2cad-479e-9a9b-345e27b47fcc None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=8386ca93fe48460ba5b6f0c10dc51122) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:47.245515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02023d75-2cad-479e-9a9b-345e27b47fcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:47.247109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02023d75-2cad-479e-9a9b-345e27b47fcc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:47.257112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 270/540] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:47 2022] GET /identity/v3/role_assignments?user.id=8386ca93fe48460ba5b6f0c10dc51122&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:47.437413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1c4c1-d33f-4cef-a0b1-5fc5840c81de None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:47.468302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:47.488120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b262b-6622-4be8-a1c6-a3cf1798ec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:47.488785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b262b-6622-4be8-a1c6-a3cf1798ec23 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:47.489071 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b262b-6622-4be8-a1c6-a3cf1798ec23 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:47.489421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b262b-6622-4be8-a1c6-a3cf1798ec23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:47.490309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b262b-6622-4be8-a1c6-a3cf1798ec23 None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=e42c796dbd7441289a7cbc328abeaf81) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:47.490627 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b262b-6622-4be8-a1c6-a3cf1798ec23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:47.492066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b262b-6622-4be8-a1c6-a3cf1798ec23 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:47.499445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 271/541] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:47 2022] GET /identity/v3/role_assignments?user.id=e42c796dbd7441289a7cbc328abeaf81&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:47.863852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea03963-e907-46a8-90db-611cb6adc008 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:47.863852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea03963-e907-46a8-90db-611cb6adc008 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:47.863852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea03963-e907-46a8-90db-611cb6adc008 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:47.863852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 271/542] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:47.871123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df6560-dfe5-4b4e-ba35-61e325df3fda None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:47.872506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df6560-dfe5-4b4e-ba35-61e325df3fda None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:47.873037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df6560-dfe5-4b4e-ba35-61e325df3fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:47.892721 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-55df6560-dfe5-4b4e-ba35-61e325df3fda None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:47.894245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-55df6560-dfe5-4b4e-ba35-61e325df3fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:47.993627 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 272/543] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:48.035456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea03963-e907-46a8-90db-611cb6adc008 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:48.050227 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3021c5-074b-47d9-8b69-2eca7b50330f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:48.050791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3021c5-074b-47d9-8b69-2eca7b50330f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:48.051244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3021c5-074b-47d9-8b69-2eca7b50330f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:48.052811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 273/544] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:48.062237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bdaf3-a0e8-452b-9793-608dd8680853 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:48.062819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bdaf3-a0e8-452b-9793-608dd8680853 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:48.063289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bdaf3-a0e8-452b-9793-608dd8680853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:48.091553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-af6bdaf3-a0e8-452b-9793-608dd8680853 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:48.093188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-af6bdaf3-a0e8-452b-9793-608dd8680853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:48.183799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea03963-e907-46a8-90db-611cb6adc008 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:48.224902 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 274/545] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:48.251411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52443e3-7fd6-416f-a25f-330640bf3a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:48.252624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52443e3-7fd6-416f-a25f-330640bf3a0b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:48.253181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52443e3-7fd6-416f-a25f-330640bf3a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:48.253654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52443e3-7fd6-416f-a25f-330640bf3a0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:48.269303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52443e3-7fd6-416f-a25f-330640bf3a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f52443e3-7fd6-416f-a25f-330640bf3a0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:48.277704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:48.279147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:48.279917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:48.282686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 272/546] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:48.289820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bdaf3-a0e8-452b-9793-608dd8680853 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:48.295898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52443e3-7fd6-416f-a25f-330640bf3a0b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:48.308442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:48.401367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:48.406823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52443e3-7fd6-416f-a25f-330640bf3a0b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:48.461258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf706c6-1604-4e4f-83b6-720b27ae783e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:48.462488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf706c6-1604-4e4f-83b6-720b27ae783e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:48.462968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf706c6-1604-4e4f-83b6-720b27ae783e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:48.463431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf706c6-1604-4e4f-83b6-720b27ae783e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:48.476972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf706c6-1604-4e4f-83b6-720b27ae783e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-dcf706c6-1604-4e4f-83b6-720b27ae783e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:48.487989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:48.488981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:48.489851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:48.498558 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 273/547] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:48.500232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf706c6-1604-4e4f-83b6-720b27ae783e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:48.557875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bdaf3-a0e8-452b-9793-608dd8680853 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:48.634054 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259c00f-3e7e-4560-9e98-df656d423f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:48.636181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259c00f-3e7e-4560-9e98-df656d423f07 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:48.638581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259c00f-3e7e-4560-9e98-df656d423f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:48.639037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259c00f-3e7e-4560-9e98-df656d423f07 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:48.640022 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259c00f-3e7e-4560-9e98-df656d423f07 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:48.640549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259c00f-3e7e-4560-9e98-df656d423f07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:48.644237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259c00f-3e7e-4560-9e98-df656d423f07 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:48.648768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:48.666992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 275/548] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:48.675356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259c00f-3e7e-4560-9e98-df656d423f07 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:48.698231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf706c6-1604-4e4f-83b6-720b27ae783e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:48.737822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff3a9-3f96-4a4d-8836-de0555fa221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:48.738738 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff3a9-3f96-4a4d-8836-de0555fa221f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:48.739175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff3a9-3f96-4a4d-8836-de0555fa221f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:48.739612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff3a9-3f96-4a4d-8836-de0555fa221f None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:48.740565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ff3a9-3f96-4a4d-8836-de0555fa221f None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:48.741034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ff3a9-3f96-4a4d-8836-de0555fa221f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:48.742630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ff3a9-3f96-4a4d-8836-de0555fa221f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:48.756689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 274/549] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:48.758304 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff3a9-3f96-4a4d-8836-de0555fa221f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:48.869494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff3a9-3f96-4a4d-8836-de0555fa221f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:48.876451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259c00f-3e7e-4560-9e98-df656d423f07 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:48.887104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:48.902295 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:48.912323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410acaa-eda3-46a9-9d7f-3653772aa1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:48.913463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410acaa-eda3-46a9-9d7f-3653772aa1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:48.913987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410acaa-eda3-46a9-9d7f-3653772aa1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:48.914476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410acaa-eda3-46a9-9d7f-3653772aa1c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:48.925986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410acaa-eda3-46a9-9d7f-3653772aa1c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:48.927834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410acaa-eda3-46a9-9d7f-3653772aa1c0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:48.934963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16357f9-fc14-48bc-8c9b-8f15ba36d210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:48.938446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16357f9-fc14-48bc-8c9b-8f15ba36d210 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:48.939064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16357f9-fc14-48bc-8c9b-8f15ba36d210 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:48.939575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16357f9-fc14-48bc-8c9b-8f15ba36d210 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f410acaa-eda3-46a9-9d7f-3653772aa1c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:48.971596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:48.972736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:48.973617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:48.973617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:48.973617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:48.973617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:48.975873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 275/550] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:48.977004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16357f9-fc14-48bc-8c9b-8f15ba36d210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:48.978714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16357f9-fc14-48bc-8c9b-8f15ba36d210 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:48.986896 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410acaa-eda3-46a9-9d7f-3653772aa1c0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f16357f9-fc14-48bc-8c9b-8f15ba36d210 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:49.018785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:49.019699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:49.020708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:49.020708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:49.020708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:49.020708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:49.034397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 276/551] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:49.035576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16357f9-fc14-48bc-8c9b-8f15ba36d210 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:49.172360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410acaa-eda3-46a9-9d7f-3653772aa1c0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:49.240064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16357f9-fc14-48bc-8c9b-8f15ba36d210 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:49.254107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f9a90-714c-441a-b680-4fbbb3239412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:49.255277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f9a90-714c-441a-b680-4fbbb3239412 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:49.255823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f9a90-714c-441a-b680-4fbbb3239412 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:49.256353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f9a90-714c-441a-b680-4fbbb3239412 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:49.257291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f9a90-714c-441a-b680-4fbbb3239412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:49.257891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f9a90-714c-441a-b680-4fbbb3239412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:49.259820 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f9a90-714c-441a-b680-4fbbb3239412 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:49.286192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 276/552] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:49.288234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f9a90-714c-441a-b680-4fbbb3239412 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:49.290983 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264d93b-9ade-4b81-bc1b-8cc8cd25aa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:49.291663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264d93b-9ade-4b81-bc1b-8cc8cd25aa82 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:49.291950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264d93b-9ade-4b81-bc1b-8cc8cd25aa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:49.292227 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264d93b-9ade-4b81-bc1b-8cc8cd25aa82 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:49.292966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9264d93b-9ade-4b81-bc1b-8cc8cd25aa82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:49.294872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9264d93b-9ade-4b81-bc1b-8cc8cd25aa82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:49.295293 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9264d93b-9ade-4b81-bc1b-8cc8cd25aa82 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:49.327447 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 277/553] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:49.330622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264d93b-9ade-4b81-bc1b-8cc8cd25aa82 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:49.354250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:49.371378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:49.484028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f9a90-714c-441a-b680-4fbbb3239412 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:49.518065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264d93b-9ade-4b81-bc1b-8cc8cd25aa82 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:49.522197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9383-d956-471c-b43d-fe439d5e7dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:49.523050 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9383-d956-471c-b43d-fe439d5e7dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:49.523339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9383-d956-471c-b43d-fe439d5e7dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:49.523621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9383-d956-471c-b43d-fe439d5e7dc0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:49.535922 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da9383-d956-471c-b43d-fe439d5e7dc0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:49.535922 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da9383-d956-471c-b43d-fe439d5e7dc0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-90da9383-d956-471c-b43d-fe439d5e7dc0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:49.551394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:49.552464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:49.553855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:49.553855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:49.553855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:49.553855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:49.553855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:49.553855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:49.553855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:49.553855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:49.553855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 16:06:49.553855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:49.553855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 277/554] 10.222.0.42 () {56 vars in 1093 bytes} [Wed May 25 16:06:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:49.560821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9383-d956-471c-b43d-fe439d5e7dc0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:49.598324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97ad6e-7e7c-47f1-bfe0-0e8d8e5e1392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:49.599129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97ad6e-7e7c-47f1-bfe0-0e8d8e5e1392 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:49.599416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97ad6e-7e7c-47f1-bfe0-0e8d8e5e1392 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:49.599697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97ad6e-7e7c-47f1-bfe0-0e8d8e5e1392 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:49.618578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97ad6e-7e7c-47f1-bfe0-0e8d8e5e1392 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:49.620057 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97ad6e-7e7c-47f1-bfe0-0e8d8e5e1392 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ad97ad6e-7e7c-47f1-bfe0-0e8d8e5e1392 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:49.648981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:49.650034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:49.651937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:49.651937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:49.651937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:49.651937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:49.651937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:49.651937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:49.651937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:49.651937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:49.651937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 16:06:49.651937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:49.658969 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 278/555] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:06:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:49.659875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97ad6e-7e7c-47f1-bfe0-0e8d8e5e1392 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:49.733454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:49.744531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9383-d956-471c-b43d-fe439d5e7dc0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:49.788396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:49.796618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3245e4a-899c-477c-b99b-800392db5564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:49.797313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3245e4a-899c-477c-b99b-800392db5564 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:49.797599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3245e4a-899c-477c-b99b-800392db5564 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:49.797933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3245e4a-899c-477c-b99b-800392db5564 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:49.798934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3245e4a-899c-477c-b99b-800392db5564 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:49.799250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3245e4a-899c-477c-b99b-800392db5564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:49.800696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3245e4a-899c-477c-b99b-800392db5564 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:49.814944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97ad6e-7e7c-47f1-bfe0-0e8d8e5e1392 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:49.824386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 278/556] 10.222.0.42 () {56 vars in 1133 bytes} [Wed May 25 16:06:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:49.826340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad16ed5-8788-4c8f-852d-b522a65e092a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:49.826637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad16ed5-8788-4c8f-852d-b522a65e092a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:49.826919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad16ed5-8788-4c8f-852d-b522a65e092a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:49.828530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 279/557] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:49.829254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad16ed5-8788-4c8f-852d-b522a65e092a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:49.852473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb43b3b-5caf-4ec4-a7eb-6755a6487555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:49.853405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43b3b-5caf-4ec4-a7eb-6755a6487555 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:49.853689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43b3b-5caf-4ec4-a7eb-6755a6487555 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:49.854375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43b3b-5caf-4ec4-a7eb-6755a6487555 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:49.855233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb43b3b-5caf-4ec4-a7eb-6755a6487555 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:49.855552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb43b3b-5caf-4ec4-a7eb-6755a6487555 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:49.858252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb43b3b-5caf-4ec4-a7eb-6755a6487555 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:49.884884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 279/558] 10.222.0.42 () {56 vars in 1135 bytes} [Wed May 25 16:06:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:49.886312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687a12-6372-417f-a965-d617e35edfca None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:49.886603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687a12-6372-417f-a965-d617e35edfca None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:49.886882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687a12-6372-417f-a965-d617e35edfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:49.910172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ac687a12-6372-417f-a965-d617e35edfca None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:49.912077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ac687a12-6372-417f-a965-d617e35edfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:49.955644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad16ed5-8788-4c8f-852d-b522a65e092a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:49.992954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 280/559] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:49.993704 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac687a12-6372-417f-a965-d617e35edfca None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:50.009576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de5659-2966-4a71-b4ce-11612f82a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:50.011004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de5659-2966-4a71-b4ce-11612f82a522 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.011503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de5659-2966-4a71-b4ce-11612f82a522 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.012338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de5659-2966-4a71-b4ce-11612f82a522 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.024987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de5659-2966-4a71-b4ce-11612f82a522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:50.026592 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de5659-2966-4a71-b4ce-11612f82a522 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-46de5659-2966-4a71-b4ce-11612f82a522 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:50.046818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:50.047948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:50.048851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:50.048851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:50.048851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:50.048851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:50.048851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 280/560] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:50.049694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4057bdf-c072-4b29-93e0-194b6a942746 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.050001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4057bdf-c072-4b29-93e0-194b6a942746 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.050278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4057bdf-c072-4b29-93e0-194b6a942746 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.051693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 281/561] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:50.054801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4057bdf-c072-4b29-93e0-194b6a942746 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:50.069258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:50.069416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:50.111533 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac687a12-6372-417f-a965-d617e35edfca None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:50.150577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f29c2-ada7-4c2a-ac39-4b054dee2642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:50.151248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f29c2-ada7-4c2a-ac39-4b054dee2642 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.151536 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f29c2-ada7-4c2a-ac39-4b054dee2642 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.151813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f29c2-ada7-4c2a-ac39-4b054dee2642 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.175052 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4057bdf-c072-4b29-93e0-194b6a942746 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:50.213914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9c969-fc9d-4660-b767-652094b8a671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:50.214725 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9c969-fc9d-4660-b767-652094b8a671 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.215007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9c969-fc9d-4660-b767-652094b8a671 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.215310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9c969-fc9d-4660-b767-652094b8a671 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.226591 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9c969-fc9d-4660-b767-652094b8a671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:50.228045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f29c2-ada7-4c2a-ac39-4b054dee2642 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:50.229480 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f29c2-ada7-4c2a-ac39-4b054dee2642 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c4e9c969-fc9d-4660-b767-652094b8a671 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:50.235023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:50.235958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:50.236799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 282/562] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:50.237490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9c969-fc9d-4660-b767-652094b8a671 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:50.303190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:50.312723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:50.346040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 281/563] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:50.346687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9c969-fc9d-4660-b767-652094b8a671 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:50.347320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000082a-7d29-4a28-b5c5-9e97ef2c190b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.347617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000082a-7d29-4a28-b5c5-9e97ef2c190b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.347894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000082a-7d29-4a28-b5c5-9e97ef2c190b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.365191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b000082a-7d29-4a28-b5c5-9e97ef2c190b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:50.366448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b000082a-7d29-4a28-b5c5-9e97ef2c190b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:50.383430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df0ee2-ca73-4a73-9703-3a45b7c67fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:50.384094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df0ee2-ca73-4a73-9703-3a45b7c67fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.384379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df0ee2-ca73-4a73-9703-3a45b7c67fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.384654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df0ee2-ca73-4a73-9703-3a45b7c67fe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.385369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df0ee2-ca73-4a73-9703-3a45b7c67fe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:50.385675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df0ee2-ca73-4a73-9703-3a45b7c67fe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:50.387072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df0ee2-ca73-4a73-9703-3a45b7c67fe5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:50.399056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 283/564] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:50.399872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df0ee2-ca73-4a73-9703-3a45b7c67fe5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:50.444378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 282/565] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:50.445735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3f2e-df19-4dbe-967f-9d3bdda86460 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.446036 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3f2e-df19-4dbe-967f-9d3bdda86460 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.446387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3f2e-df19-4dbe-967f-9d3bdda86460 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.447566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 283/566] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:50.449873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff3f2e-df19-4dbe-967f-9d3bdda86460 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:50.505963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df0ee2-ca73-4a73-9703-3a45b7c67fe5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:50.535097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:50.541950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589cd2f-e9ff-45b1-8faa-5f8a265aa63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:50.542627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589cd2f-e9ff-45b1-8faa-5f8a265aa63a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.542915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589cd2f-e9ff-45b1-8faa-5f8a265aa63a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.543193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589cd2f-e9ff-45b1-8faa-5f8a265aa63a None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.544557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589cd2f-e9ff-45b1-8faa-5f8a265aa63a None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:50.544866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589cd2f-e9ff-45b1-8faa-5f8a265aa63a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:50.546271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589cd2f-e9ff-45b1-8faa-5f8a265aa63a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:50.553088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:50.554272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 284/567] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:50.555075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589cd2f-e9ff-45b1-8faa-5f8a265aa63a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:50.565451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff3f2e-df19-4dbe-967f-9d3bdda86460 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:50.604891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a200760-8d1f-43ec-82c2-e4d530246c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:50.605592 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a200760-8d1f-43ec-82c2-e4d530246c98 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.605936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a200760-8d1f-43ec-82c2-e4d530246c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.607108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a200760-8d1f-43ec-82c2-e4d530246c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.617216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a200760-8d1f-43ec-82c2-e4d530246c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:50.618731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a200760-8d1f-43ec-82c2-e4d530246c98 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0a200760-8d1f-43ec-82c2-e4d530246c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:50.636097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:50.637015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:50.638439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:50.638439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:50.638439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:50.638439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:50.638439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 284/568] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:50.639270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b41d4d-ec2b-44e5-b1cc-9fa7a6a8def4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.639466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b41d4d-ec2b-44e5-b1cc-9fa7a6a8def4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.639753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b41d4d-ec2b-44e5-b1cc-9fa7a6a8def4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.656203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f0b41d4d-ec2b-44e5-b1cc-9fa7a6a8def4 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:50.657404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f0b41d4d-ec2b-44e5-b1cc-9fa7a6a8def4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:50.659570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589cd2f-e9ff-45b1-8faa-5f8a265aa63a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:50.692984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b004925-d14d-41fb-b885-9b4dadf0a916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:50.693818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b004925-d14d-41fb-b885-9b4dadf0a916 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.694103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b004925-d14d-41fb-b885-9b4dadf0a916 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.694379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b004925-d14d-41fb-b885-9b4dadf0a916 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.701885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b004925-d14d-41fb-b885-9b4dadf0a916 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:50.703336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b004925-d14d-41fb-b885-9b4dadf0a916 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4b004925-d14d-41fb-b885-9b4dadf0a916 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:06:50.717394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:50.718400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:50.719726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:06:50.719726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:06:50.719726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:06:50.719726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:06:50.719726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:06:50.719726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:06:50.719726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:50.719726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:50.719726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 285/569] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:50.719726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b004925-d14d-41fb-b885-9b4dadf0a916 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:50.730146 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 285/570] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:50.731552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b41d4d-ec2b-44e5-b1cc-9fa7a6a8def4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:50.782224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:50.808770 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:50.831027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b004925-d14d-41fb-b885-9b4dadf0a916 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:50.832333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b41d4d-ec2b-44e5-b1cc-9fa7a6a8def4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:50.865465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c78d51-401f-4251-9f6c-bb36645e63a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:50.865918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe1344-ca55-404b-9e8e-279df1cb1018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:50.866218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c78d51-401f-4251-9f6c-bb36645e63a3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.866496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c78d51-401f-4251-9f6c-bb36645e63a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.866714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe1344-ca55-404b-9e8e-279df1cb1018 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:50.866813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c78d51-401f-4251-9f6c-bb36645e63a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.866996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe1344-ca55-404b-9e8e-279df1cb1018 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:50.867273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe1344-ca55-404b-9e8e-279df1cb1018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:50.877115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe1344-ca55-404b-9e8e-279df1cb1018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:50.877402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c78d51-401f-4251-9f6c-bb36645e63a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:50.878603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe1344-ca55-404b-9e8e-279df1cb1018 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f3c78d51-401f-4251-9f6c-bb36645e63a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:50.885164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:50.886266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:50.887113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 286/571] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:50.888392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c78d51-401f-4251-9f6c-bb36645e63a3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-fafe1344-ca55-404b-9e8e-279df1cb1018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:50.901336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:50.902610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:50.904695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:50.904695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:50.904695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:50.904695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:50.904695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 286/572] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:50.904695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe1344-ca55-404b-9e8e-279df1cb1018 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:50.990089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c78d51-401f-4251-9f6c-bb36645e63a3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.003428 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:51.005915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe1344-ca55-404b-9e8e-279df1cb1018 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.024648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab72d21-01a0-4aa0-b8a4-9263e281fc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.025305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab72d21-01a0-4aa0-b8a4-9263e281fc6f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.025602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab72d21-01a0-4aa0-b8a4-9263e281fc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.025916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab72d21-01a0-4aa0-b8a4-9263e281fc6f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.026638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab72d21-01a0-4aa0-b8a4-9263e281fc6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:51.026944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab72d21-01a0-4aa0-b8a4-9263e281fc6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:51.028810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab72d21-01a0-4aa0-b8a4-9263e281fc6f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:51.033191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:51.041429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 287/573] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:51.041602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac05dfc-da52-4d85-8cf5-57b849c08823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.042214 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac05dfc-da52-4d85-8cf5-57b849c08823 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.042313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab72d21-01a0-4aa0-b8a4-9263e281fc6f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.042500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac05dfc-da52-4d85-8cf5-57b849c08823 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.042773 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac05dfc-da52-4d85-8cf5-57b849c08823 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.044199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac05dfc-da52-4d85-8cf5-57b849c08823 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:51.044512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac05dfc-da52-4d85-8cf5-57b849c08823 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:51.045975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac05dfc-da52-4d85-8cf5-57b849c08823 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:51.057608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 287/574] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:06:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:51.058458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac05dfc-da52-4d85-8cf5-57b849c08823 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.149599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab72d21-01a0-4aa0-b8a4-9263e281fc6f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.157694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac05dfc-da52-4d85-8cf5-57b849c08823 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.183525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d40cf7-c023-43fd-821c-12db8aa0298b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.184868 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d40cf7-c023-43fd-821c-12db8aa0298b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.185152 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d40cf7-c023-43fd-821c-12db8aa0298b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.185426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d40cf7-c023-43fd-821c-12db8aa0298b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.192145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3bd6e-d093-4a40-95fb-dbc5e14ded2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.192811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3bd6e-d093-4a40-95fb-dbc5e14ded2c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.193103 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3bd6e-d093-4a40-95fb-dbc5e14ded2c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.193380 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3bd6e-d093-4a40-95fb-dbc5e14ded2c None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.194164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3bd6e-d093-4a40-95fb-dbc5e14ded2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:51.194474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3bd6e-d093-4a40-95fb-dbc5e14ded2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:51.195812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d40cf7-c023-43fd-821c-12db8aa0298b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:51.195919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3bd6e-d093-4a40-95fb-dbc5e14ded2c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f2d40cf7-c023-43fd-821c-12db8aa0298b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:51.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:51.204998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:51.210538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:51.211333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 288/575] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:51.211333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3bd6e-d093-4a40-95fb-dbc5e14ded2c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.211496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 288/576] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:51.211496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d40cf7-c023-43fd-821c-12db8aa0298b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.241372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:51.258179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:51.312857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d40cf7-c023-43fd-821c-12db8aa0298b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.313063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3bd6e-d093-4a40-95fb-dbc5e14ded2c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.346769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01121d8a-e396-4712-ade7-57169b448689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.347108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3072b-5a4a-415e-8dae-a415f080e5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.347506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01121d8a-e396-4712-ade7-57169b448689 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.347716 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01121d8a-e396-4712-ade7-57169b448689 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.347905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3072b-5a4a-415e-8dae-a415f080e5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.347996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01121d8a-e396-4712-ade7-57169b448689 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.348191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3072b-5a4a-415e-8dae-a415f080e5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.348505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3072b-5a4a-415e-8dae-a415f080e5b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.348698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01121d8a-e396-4712-ade7-57169b448689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:51.349004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01121d8a-e396-4712-ade7-57169b448689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:51.350416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01121d8a-e396-4712-ade7-57169b448689 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:51.355237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3072b-5a4a-415e-8dae-a415f080e5b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:51.357268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3072b-5a4a-415e-8dae-a415f080e5b9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:51.362729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 289/577] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:51.363636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01121d8a-e396-4712-ade7-57169b448689 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a1e3072b-5a4a-415e-8dae-a415f080e5b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:06:51.372523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:51.373602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:51.374670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:06:51.374670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:06:51.374670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:06:51.374670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:06:51.374670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:06:51.374670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:06:51.374670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:51.374670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:51.374670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 289/578] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:51.375236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3072b-5a4a-415e-8dae-a415f080e5b9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.470268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:51.472758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01121d8a-e396-4712-ade7-57169b448689 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.479059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3072b-5a4a-415e-8dae-a415f080e5b9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.484457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:51.508647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6fd88-599b-4ea8-a03f-6da1ba96886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.509536 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6fd88-599b-4ea8-a03f-6da1ba96886f None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.509854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6fd88-599b-4ea8-a03f-6da1ba96886f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.510135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6fd88-599b-4ea8-a03f-6da1ba96886f None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/6e6795b5b5cf4499ad494b875756ba71/roles/6bc01dbf83c844fe802466d3c4ada6af` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.513979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e7ca0-de31-438c-a4a2-7147e38d418f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.514774 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e7ca0-de31-438c-a4a2-7147e38d418f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.515052 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e7ca0-de31-438c-a4a2-7147e38d418f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.515321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e7ca0-de31-438c-a4a2-7147e38d418f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.525688 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e7ca0-de31-438c-a4a2-7147e38d418f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:51.526181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6fd88-599b-4ea8-a03f-6da1ba96886f None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=6e6795b5b5cf4499ad494b875756ba71, role_id=6bc01dbf83c844fe802466d3c4ada6af)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:51.527161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e7ca0-de31-438c-a4a2-7147e38d418f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:51.527618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6fd88-599b-4ea8-a03f-6da1ba96886f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:51.543267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 290/579] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:06:51 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/6e6795b5b5cf4499ad494b875756ba71/roles/6bc01dbf83c844fe802466d3c4ada6af => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:06:51.544940 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6fd88-599b-4ea8-a03f-6da1ba96886f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-789e7ca0-de31-438c-a4a2-7147e38d418f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:51.547114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:51.548045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:51.554587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:51.554587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:51.554587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:51.554587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:51.554587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 290/580] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:51.554587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e7ca0-de31-438c-a4a2-7147e38d418f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.660556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6fd88-599b-4ea8-a03f-6da1ba96886f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.677091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e7ca0-de31-438c-a4a2-7147e38d418f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.699457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599554d-63bc-45e7-b28e-ec0fb4616722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.700089 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599554d-63bc-45e7-b28e-ec0fb4616722 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.700486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599554d-63bc-45e7-b28e-ec0fb4616722 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.700786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599554d-63bc-45e7-b28e-ec0fb4616722 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.701511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599554d-63bc-45e7-b28e-ec0fb4616722 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:51.701848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599554d-63bc-45e7-b28e-ec0fb4616722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:51.703256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599554d-63bc-45e7-b28e-ec0fb4616722 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:51.712928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 291/581] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:51.713923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599554d-63bc-45e7-b28e-ec0fb4616722 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.722397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4a96d-9c56-4169-ab53-410bfd7df60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.723216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a96d-9c56-4169-ab53-410bfd7df60f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.723505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a96d-9c56-4169-ab53-410bfd7df60f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.723786 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a96d-9c56-4169-ab53-410bfd7df60f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.735170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4a96d-9c56-4169-ab53-410bfd7df60f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:51.736641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4a96d-9c56-4169-ab53-410bfd7df60f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:51.746670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ffc4a96d-9c56-4169-ab53-410bfd7df60f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:51.756332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:51.757336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:51.758296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:51.758296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:51.758296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:51.758296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:51.758296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:51.758296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:51.758296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:51.758296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:51.758296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 16:06:51.758296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:51.758296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 291/582] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:51.758885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4a96d-9c56-4169-ab53-410bfd7df60f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.763755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:51.844925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599554d-63bc-45e7-b28e-ec0fb4616722 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.873842 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4a96d-9c56-4169-ab53-410bfd7df60f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:51.879878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666f620-15a9-4aa5-86d1-022ea2a873b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.881787 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666f620-15a9-4aa5-86d1-022ea2a873b3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.882847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666f620-15a9-4aa5-86d1-022ea2a873b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.883535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666f620-15a9-4aa5-86d1-022ea2a873b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.884422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666f620-15a9-4aa5-86d1-022ea2a873b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:51.884796 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666f620-15a9-4aa5-86d1-022ea2a873b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:51.886423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666f620-15a9-4aa5-86d1-022ea2a873b3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:51.905814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 292/583] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:51.905814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666f620-15a9-4aa5-86d1-022ea2a873b3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.919262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83b0d1-0b35-4c5b-ac0a-9e0e86e319a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:51.919935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83b0d1-0b35-4c5b-ac0a-9e0e86e319a0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:51.920225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83b0d1-0b35-4c5b-ac0a-9e0e86e319a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:51.920502 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83b0d1-0b35-4c5b-ac0a-9e0e86e319a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:51.921237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83b0d1-0b35-4c5b-ac0a-9e0e86e319a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:51.921544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83b0d1-0b35-4c5b-ac0a-9e0e86e319a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:51.922950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83b0d1-0b35-4c5b-ac0a-9e0e86e319a0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:51.939928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 292/584] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:51.939928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83b0d1-0b35-4c5b-ac0a-9e0e86e319a0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:51.990893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:52.008502 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:52.012139 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666f620-15a9-4aa5-86d1-022ea2a873b3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:52.048789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50f274-d5e4-4741-866b-9ef334c9587c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:52.049591 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50f274-d5e4-4741-866b-9ef334c9587c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:52.049935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50f274-d5e4-4741-866b-9ef334c9587c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:52.050223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50f274-d5e4-4741-866b-9ef334c9587c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:52.055302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83b0d1-0b35-4c5b-ac0a-9e0e86e319a0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:52.060900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50f274-d5e4-4741-866b-9ef334c9587c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:52.062400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50f274-d5e4-4741-866b-9ef334c9587c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ac50f274-d5e4-4741-866b-9ef334c9587c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:52.082502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:52.083640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:52.084540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:52.084540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:52.084540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:52.084540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:52.084540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 293/585] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:52.084540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50f274-d5e4-4741-866b-9ef334c9587c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:52.095150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a081ca-ef20-4d14-be63-48efb9d195b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:52.095836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a081ca-ef20-4d14-be63-48efb9d195b1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:52.096118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a081ca-ef20-4d14-be63-48efb9d195b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:52.096394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a081ca-ef20-4d14-be63-48efb9d195b1 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:52.097245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a081ca-ef20-4d14-be63-48efb9d195b1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:52.097556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a081ca-ef20-4d14-be63-48efb9d195b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:52.099174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a081ca-ef20-4d14-be63-48efb9d195b1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:52.119302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 293/586] 10.222.0.42 () {56 vars in 1137 bytes} [Wed May 25 16:06:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:52.120551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a081ca-ef20-4d14-be63-48efb9d195b1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:52.208341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50f274-d5e4-4741-866b-9ef334c9587c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:52.230740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a081ca-ef20-4d14-be63-48efb9d195b1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:52.248287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d430d-27f2-44de-a5e8-df5c17ae1aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:52.249434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d430d-27f2-44de-a5e8-df5c17ae1aef None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:52.249745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d430d-27f2-44de-a5e8-df5c17ae1aef None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:52.250018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d430d-27f2-44de-a5e8-df5c17ae1aef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:52.250940 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d430d-27f2-44de-a5e8-df5c17ae1aef None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=c6eb13611d0d4b6abe035c9bfbfde79c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:52.251254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d430d-27f2-44de-a5e8-df5c17ae1aef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:52.252835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d430d-27f2-44de-a5e8-df5c17ae1aef None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:52.255222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:52.255377 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:52.261491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 294/587] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:52 2022] GET /identity/v3/role_assignments?user.id=c6eb13611d0d4b6abe035c9bfbfde79c&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:52.262827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d430d-27f2-44de-a5e8-df5c17ae1aef None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:52.270990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269083a-743e-4bc9-a97f-9c02a1c18997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:52.272264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269083a-743e-4bc9-a97f-9c02a1c18997 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:52.272772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269083a-743e-4bc9-a97f-9c02a1c18997 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:52.273298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269083a-743e-4bc9-a97f-9c02a1c18997 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:52.282339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269083a-743e-4bc9-a97f-9c02a1c18997 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:52.284487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269083a-743e-4bc9-a97f-9c02a1c18997 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d269083a-743e-4bc9-a97f-9c02a1c18997 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:52.301846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:52.302859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:52.303797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:52.303797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:52.303797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:52.303797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:52.303797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:52.303797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:52.303797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:52.303797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:52.303797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 16:06:52.303797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:52.306604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 294/588] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:52.307741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269083a-743e-4bc9-a97f-9c02a1c18997 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:52.387742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d430d-27f2-44de-a5e8-df5c17ae1aef None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:52.423634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269083a-743e-4bc9-a97f-9c02a1c18997 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:52.431897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edfc6a-6bd2-4317-acb4-26e5d14524bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:52.432660 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edfc6a-6bd2-4317-acb4-26e5d14524bc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:52.432944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edfc6a-6bd2-4317-acb4-26e5d14524bc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:52.435817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edfc6a-6bd2-4317-acb4-26e5d14524bc None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:52.436239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edfc6a-6bd2-4317-acb4-26e5d14524bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:52.436563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edfc6a-6bd2-4317-acb4-26e5d14524bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:52.438135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edfc6a-6bd2-4317-acb4-26e5d14524bc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:52.458161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 295/589] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:52.458161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edfc6a-6bd2-4317-acb4-26e5d14524bc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:52.478825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d9b74-8e47-44f2-b4a5-8550b5584a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:52.479631 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9b74-8e47-44f2-b4a5-8550b5584a63 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:52.479916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9b74-8e47-44f2-b4a5-8550b5584a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:52.480191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9b74-8e47-44f2-b4a5-8550b5584a63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:52.497070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d9b74-8e47-44f2-b4a5-8550b5584a63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:52.498611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d9b74-8e47-44f2-b4a5-8550b5584a63 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-850d9b74-8e47-44f2-b4a5-8550b5584a63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:52.522424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:52.523382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:52.524275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:52.524275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:52.524275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:52.524275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:52.524275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 295/590] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:52.525017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d9b74-8e47-44f2-b4a5-8550b5584a63 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:52.550123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:52.567754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:52.610870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edfc6a-6bd2-4317-acb4-26e5d14524bc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:52.643155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d9b74-8e47-44f2-b4a5-8550b5584a63 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:52.656262 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390dab5f-70fe-4e5d-acc3-9a6aca526815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:52.656980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dab5f-70fe-4e5d-acc3-9a6aca526815 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:52.657506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dab5f-70fe-4e5d-acc3-9a6aca526815 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:52.657823 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dab5f-70fe-4e5d-acc3-9a6aca526815 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:52.658915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390dab5f-70fe-4e5d-acc3-9a6aca526815 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:52.659235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390dab5f-70fe-4e5d-acc3-9a6aca526815 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:52.661189 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390dab5f-70fe-4e5d-acc3-9a6aca526815 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:52.684387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 296/591] 10.222.0.42 () {56 vars in 1137 bytes} [Wed May 25 16:06:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:52.687742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a8f99-b35b-44fb-a74c-ff8cfe85478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:52.688866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a8f99-b35b-44fb-a74c-ff8cfe85478d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:52.689599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a8f99-b35b-44fb-a74c-ff8cfe85478d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:52.689862 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390dab5f-70fe-4e5d-acc3-9a6aca526815 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:52.690486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a8f99-b35b-44fb-a74c-ff8cfe85478d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:52.700146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a8f99-b35b-44fb-a74c-ff8cfe85478d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:52.702186 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a8f99-b35b-44fb-a74c-ff8cfe85478d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-841a8f99-b35b-44fb-a74c-ff8cfe85478d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:52.718986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:52.720008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:52.720938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:52.720938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:52.720938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:52.720938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:52.720938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:52.720938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:52.720938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:52.720938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:52.720938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 16:06:52.720938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:52.728072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 296/592] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:52.729245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a8f99-b35b-44fb-a74c-ff8cfe85478d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:52.814435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390dab5f-70fe-4e5d-acc3-9a6aca526815 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:52.836553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:52.837988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:52.842184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a8f99-b35b-44fb-a74c-ff8cfe85478d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:52.862343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bb874-cba5-452a-bc09-a76c957375f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:52.863265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bb874-cba5-452a-bc09-a76c957375f9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:52.863865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bb874-cba5-452a-bc09-a76c957375f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:52.864288 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bb874-cba5-452a-bc09-a76c957375f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:52.865554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bb874-cba5-452a-bc09-a76c957375f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:52.866037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bb874-cba5-452a-bc09-a76c957375f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:52.867593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bb874-cba5-452a-bc09-a76c957375f9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:52.882871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12767bec-448a-43dd-9e8b-b637ee5b72bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:52.888119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12767bec-448a-43dd-9e8b-b637ee5b72bf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:52.888119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12767bec-448a-43dd-9e8b-b637ee5b72bf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:52.888119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12767bec-448a-43dd-9e8b-b637ee5b72bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:52.889305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 297/593] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:52.890877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bb874-cba5-452a-bc09-a76c957375f9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:52.899553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12767bec-448a-43dd-9e8b-b637ee5b72bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:52.901122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12767bec-448a-43dd-9e8b-b637ee5b72bf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-12767bec-448a-43dd-9e8b-b637ee5b72bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:52.927270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:52.928433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:52.929546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:52.929546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:52.929546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:52.929546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:52.935405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 297/594] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:52.935405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12767bec-448a-43dd-9e8b-b637ee5b72bf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:53.013148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bb874-cba5-452a-bc09-a76c957375f9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:53.050040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6b506-fb21-4c80-958a-765264088ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:53.051095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6b506-fb21-4c80-958a-765264088ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:53.051217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6b506-fb21-4c80-958a-765264088ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:53.051575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6b506-fb21-4c80-958a-765264088ac1 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:53.052528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6b506-fb21-4c80-958a-765264088ac1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:53.052931 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6b506-fb21-4c80-958a-765264088ac1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:53.054504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6b506-fb21-4c80-958a-765264088ac1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:53.066326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12767bec-448a-43dd-9e8b-b637ee5b72bf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:53.078260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 298/595] 10.222.0.42 () {56 vars in 1139 bytes} [Wed May 25 16:06:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:53.079384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6b506-fb21-4c80-958a-765264088ac1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:53.116501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe3e0-88f5-4c29-9ca2-09d204d26f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:53.117309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe3e0-88f5-4c29-9ca2-09d204d26f99 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:53.117591 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe3e0-88f5-4c29-9ca2-09d204d26f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:53.117891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe3e0-88f5-4c29-9ca2-09d204d26f99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:53.127537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe3e0-88f5-4c29-9ca2-09d204d26f99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:53.128979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe3e0-88f5-4c29-9ca2-09d204d26f99 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:53.132381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-977fe3e0-88f5-4c29-9ca2-09d204d26f99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:06:53.145939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:53.146900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:53.148393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:06:53.148393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:06:53.148393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:06:53.148393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:06:53.148393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:06:53.148393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:06:53.148393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:53.148393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:53.148393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 298/596] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:53.148393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe3e0-88f5-4c29-9ca2-09d204d26f99 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:53.159059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:53.226668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6b506-fb21-4c80-958a-765264088ac1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:53.268432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce823059-a6d8-4d44-a180-111965a78420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:53.269495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce823059-a6d8-4d44-a180-111965a78420 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:53.270120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce823059-a6d8-4d44-a180-111965a78420 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:53.270631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce823059-a6d8-4d44-a180-111965a78420 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:53.271610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce823059-a6d8-4d44-a180-111965a78420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:53.272126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce823059-a6d8-4d44-a180-111965a78420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:53.273966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce823059-a6d8-4d44-a180-111965a78420 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:53.276314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe3e0-88f5-4c29-9ca2-09d204d26f99 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:53.288182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 299/597] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:53.289468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce823059-a6d8-4d44-a180-111965a78420 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:53.326526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fed36c-55df-441b-8bd1-f11cd13bc9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:53.327381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fed36c-55df-441b-8bd1-f11cd13bc9bb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:53.327792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fed36c-55df-441b-8bd1-f11cd13bc9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:53.328083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fed36c-55df-441b-8bd1-f11cd13bc9bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:53.342801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fed36c-55df-441b-8bd1-f11cd13bc9bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:53.344537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fed36c-55df-441b-8bd1-f11cd13bc9bb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-97fed36c-55df-441b-8bd1-f11cd13bc9bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:53.369184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:53.370396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:53.371265 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:53.371265 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:53.371265 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:53.371265 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:53.373862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 299/598] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:53.375221 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fed36c-55df-441b-8bd1-f11cd13bc9bb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:53.406728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:53.415702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce823059-a6d8-4d44-a180-111965a78420 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:53.460466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:53.468383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a9355-862a-4347-b57b-0609c56d2cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:53.469191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a9355-862a-4347-b57b-0609c56d2cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:53.469497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a9355-862a-4347-b57b-0609c56d2cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:53.469918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a9355-862a-4347-b57b-0609c56d2cc3 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:53.470721 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a9355-862a-4347-b57b-0609c56d2cc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:53.471034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a9355-862a-4347-b57b-0609c56d2cc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:53.472471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a9355-862a-4347-b57b-0609c56d2cc3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:53.490331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 300/599] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:06:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:53.491767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a9355-862a-4347-b57b-0609c56d2cc3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:53.514979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fed36c-55df-441b-8bd1-f11cd13bc9bb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:53.560566 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a414eb3-ade5-4e1a-b238-ad8e9d26c4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:53.561386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a414eb3-ade5-4e1a-b238-ad8e9d26c4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:53.561670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a414eb3-ade5-4e1a-b238-ad8e9d26c4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:53.561977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a414eb3-ade5-4e1a-b238-ad8e9d26c4d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:53.570129 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a414eb3-ade5-4e1a-b238-ad8e9d26c4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:53.571699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a414eb3-ade5-4e1a-b238-ad8e9d26c4d2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5a414eb3-ade5-4e1a-b238-ad8e9d26c4d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:06:53.587815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:53.588748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:53.590070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:06:53.590070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:06:53.590070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:06:53.590070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:06:53.590070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:06:53.590070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:06:53.590070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:53.590070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:53.590070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 300/600] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:53.591017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a414eb3-ade5-4e1a-b238-ad8e9d26c4d2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:53.622766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a9355-862a-4347-b57b-0609c56d2cc3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:53.671098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ada9f5-8974-4d0f-8f7d-9ee7d946840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:53.672085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ada9f5-8974-4d0f-8f7d-9ee7d946840f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:53.673411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ada9f5-8974-4d0f-8f7d-9ee7d946840f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:53.673748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ada9f5-8974-4d0f-8f7d-9ee7d946840f None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:53.674497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ada9f5-8974-4d0f-8f7d-9ee7d946840f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:53.674809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ada9f5-8974-4d0f-8f7d-9ee7d946840f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:53.676302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ada9f5-8974-4d0f-8f7d-9ee7d946840f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:53.693478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 301/601] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:53.694372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ada9f5-8974-4d0f-8f7d-9ee7d946840f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:53.714695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a414eb3-ade5-4e1a-b238-ad8e9d26c4d2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:53.715058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:53.739429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:53.765230 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9695a9-e82b-44cc-abd3-95e84e98763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:53.766419 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9695a9-e82b-44cc-abd3-95e84e98763b None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:53.766712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9695a9-e82b-44cc-abd3-95e84e98763b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:53.766989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9695a9-e82b-44cc-abd3-95e84e98763b None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/b9913b9eb502434985f84c4a11632ccc/roles/6bc01dbf83c844fe802466d3c4ada6af` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:53.788321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9695a9-e82b-44cc-abd3-95e84e98763b None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=b9913b9eb502434985f84c4a11632ccc, role_id=6bc01dbf83c844fe802466d3c4ada6af)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:53.790538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9695a9-e82b-44cc-abd3-95e84e98763b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:53.811111 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ada9f5-8974-4d0f-8f7d-9ee7d946840f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:53.813789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 301/602] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:06:53 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/b9913b9eb502434985f84c4a11632ccc/roles/6bc01dbf83c844fe802466d3c4ada6af => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:06:53.813789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9695a9-e82b-44cc-abd3-95e84e98763b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:53.882042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129fea4-8f73-4cd1-834e-c040d0f66549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:53.882635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129fea4-8f73-4cd1-834e-c040d0f66549 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:53.883327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129fea4-8f73-4cd1-834e-c040d0f66549 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:53.883446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129fea4-8f73-4cd1-834e-c040d0f66549 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:53.884244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129fea4-8f73-4cd1-834e-c040d0f66549 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:53.884649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129fea4-8f73-4cd1-834e-c040d0f66549 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:53.886329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129fea4-8f73-4cd1-834e-c040d0f66549 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:53.902963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 302/603] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:06:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:53.907875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480e83e-3d58-457d-94bc-b75e41856f33 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:53.907875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480e83e-3d58-457d-94bc-b75e41856f33 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:53.907875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480e83e-3d58-457d-94bc-b75e41856f33 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:53.911463 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 303/604] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:53.912654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480e83e-3d58-457d-94bc-b75e41856f33 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:53.942363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9695a9-e82b-44cc-abd3-95e84e98763b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:53.981445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98b5c0-9008-422c-9ee6-40d0c425aab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:53.982685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98b5c0-9008-422c-9ee6-40d0c425aab6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:53.983146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98b5c0-9008-422c-9ee6-40d0c425aab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:53.983569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98b5c0-9008-422c-9ee6-40d0c425aab6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:53.991996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98b5c0-9008-422c-9ee6-40d0c425aab6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:53.993921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98b5c0-9008-422c-9ee6-40d0c425aab6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:54.007950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8a98b5c0-9008-422c-9ee6-40d0c425aab6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:06:54.011309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:54.012777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:54.013971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:06:54.013971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:06:54.013971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:06:54.013971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:06:54.013971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:06:54.013971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:06:54.013971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:54.013971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:54.020569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 302/605] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:54.022335 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c7665-74b8-4dfc-a637-39382b8d1971 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:54.022841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c7665-74b8-4dfc-a637-39382b8d1971 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:54.023302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c7665-74b8-4dfc-a637-39382b8d1971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:54.045280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c08c7665-74b8-4dfc-a637-39382b8d1971 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:54.046819 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c08c7665-74b8-4dfc-a637-39382b8d1971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:54.047080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480e83e-3d58-457d-94bc-b75e41856f33 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:54.085301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e8252-716f-4f6b-a340-78e73123343d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:54.086552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e8252-716f-4f6b-a340-78e73123343d None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:54.086904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e8252-716f-4f6b-a340-78e73123343d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:54.087300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e8252-716f-4f6b-a340-78e73123343d None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/8386ca93fe48460ba5b6f0c10dc51122/roles/6bc01dbf83c844fe802466d3c4ada6af` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:54.090168 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:54.104007 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e8252-716f-4f6b-a340-78e73123343d None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=8386ca93fe48460ba5b6f0c10dc51122, role_id=6bc01dbf83c844fe802466d3c4ada6af)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:54.105350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e8252-716f-4f6b-a340-78e73123343d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:54.126039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 304/606] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:06:53 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/8386ca93fe48460ba5b6f0c10dc51122/roles/6bc01dbf83c844fe802466d3c4ada6af => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:06:54.133827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e8252-716f-4f6b-a340-78e73123343d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:54.148964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 303/607] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:54.187724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c7665-74b8-4dfc-a637-39382b8d1971 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:54.256959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e8252-716f-4f6b-a340-78e73123343d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:54.280128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:54.292881 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7609ae6-22fe-4e7c-8f50-44f44129c316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:54.294810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609ae6-22fe-4e7c-8f50-44f44129c316 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:54.295339 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609ae6-22fe-4e7c-8f50-44f44129c316 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:54.295822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609ae6-22fe-4e7c-8f50-44f44129c316 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:54.296887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7609ae6-22fe-4e7c-8f50-44f44129c316 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:54.297406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7609ae6-22fe-4e7c-8f50-44f44129c316 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:54.299253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7609ae6-22fe-4e7c-8f50-44f44129c316 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:54.313513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 305/608] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:06:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:54.323516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7609ae6-22fe-4e7c-8f50-44f44129c316 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:54.353359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c7665-74b8-4dfc-a637-39382b8d1971 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:54.393530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cbd61-abb0-4bca-b335-1e49a5eac33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:54.393873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cbd61-abb0-4bca-b335-1e49a5eac33b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:54.394167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cbd61-abb0-4bca-b335-1e49a5eac33b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:54.394446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cbd61-abb0-4bca-b335-1e49a5eac33b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:54.407216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2cbd61-abb0-4bca-b335-1e49a5eac33b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1e2cbd61-abb0-4bca-b335-1e49a5eac33b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:54.415771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:54.416773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:54.417634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 304/609] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:54.423956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cbd61-abb0-4bca-b335-1e49a5eac33b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:54.451351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:54.539164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7609ae6-22fe-4e7c-8f50-44f44129c316 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:54.577360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cbd61-abb0-4bca-b335-1e49a5eac33b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:54.580106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d97e6-9586-44e4-864f-b83dbb82c8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:54.581336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d97e6-9586-44e4-864f-b83dbb82c8bb None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:54.581926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d97e6-9586-44e4-864f-b83dbb82c8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:54.582406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d97e6-9586-44e4-864f-b83dbb82c8bb None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/e42c796dbd7441289a7cbc328abeaf81/roles/6bc01dbf83c844fe802466d3c4ada6af` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:54.598716 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d97e6-9586-44e4-864f-b83dbb82c8bb None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=e42c796dbd7441289a7cbc328abeaf81, role_id=6bc01dbf83c844fe802466d3c4ada6af)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:54.600516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d97e6-9586-44e4-864f-b83dbb82c8bb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:54.618058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 306/610] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:06:54 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/e42c796dbd7441289a7cbc328abeaf81/roles/6bc01dbf83c844fe802466d3c4ada6af => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:06:54.618426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac7035-f70d-4d0c-a64f-57e83e77458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:54.619158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7035-f70d-4d0c-a64f-57e83e77458a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:54.627991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7035-f70d-4d0c-a64f-57e83e77458a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:54.627991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7035-f70d-4d0c-a64f-57e83e77458a None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:54.627991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac7035-f70d-4d0c-a64f-57e83e77458a None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:54.627991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac7035-f70d-4d0c-a64f-57e83e77458a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:54.628912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac7035-f70d-4d0c-a64f-57e83e77458a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:54.648524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 305/611] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:54.654667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d97e6-9586-44e4-864f-b83dbb82c8bb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:54.669790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022db953-efb5-4cca-8801-adff4c40fead None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:54.669790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022db953-efb5-4cca-8801-adff4c40fead None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:54.669790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022db953-efb5-4cca-8801-adff4c40fead None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:54.674544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 306/612] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:54.676397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1a1c1-21c5-4ae7-a918-a4a01d29626e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:54.676693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1a1c1-21c5-4ae7-a918-a4a01d29626e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:54.676978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1a1c1-21c5-4ae7-a918-a4a01d29626e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:54.717092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-daa1a1c1-21c5-4ae7-a918-a4a01d29626e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:54.719551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-daa1a1c1-21c5-4ae7-a918-a4a01d29626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:54.736647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:54.814592 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d97e6-9586-44e4-864f-b83dbb82c8bb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:54.853684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f642e-c698-4bd1-8f59-b2a29bdd2280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:54.854618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f642e-c698-4bd1-8f59-b2a29bdd2280 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:54.854906 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f642e-c698-4bd1-8f59-b2a29bdd2280 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:54.855236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f642e-c698-4bd1-8f59-b2a29bdd2280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:54.868053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f642e-c698-4bd1-8f59-b2a29bdd2280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:54.869559 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f642e-c698-4bd1-8f59-b2a29bdd2280 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5d2f642e-c698-4bd1-8f59-b2a29bdd2280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:54.890442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:54.891395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:54.892285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:54.892285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:54.892285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:54.892285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:54.892285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 307/613] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:54.897131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f642e-c698-4bd1-8f59-b2a29bdd2280 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:54.904429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 307/614] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:54.934344 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1a1c1-21c5-4ae7-a918-a4a01d29626e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:54.949856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:55.030853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:55.055153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1a1c1-21c5-4ae7-a918-a4a01d29626e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:55.092417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e0ade-0fe2-4a7a-9496-dfff850c3119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:55.094092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e0ade-0fe2-4a7a-9496-dfff850c3119 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:55.094604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e0ade-0fe2-4a7a-9496-dfff850c3119 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:55.095083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e0ade-0fe2-4a7a-9496-dfff850c3119 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:55.106341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e0ade-0fe2-4a7a-9496-dfff850c3119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:55.111885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f642e-c698-4bd1-8f59-b2a29bdd2280 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-de1e0ade-0fe2-4a7a-9496-dfff850c3119 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:55.115637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:55.116565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:55.117408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:55.124958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 308/615] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:55.126748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e0ade-0fe2-4a7a-9496-dfff850c3119 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:55.186151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac648c6-19e5-4743-8747-6d74ecc24d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:55.186746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac648c6-19e5-4743-8747-6d74ecc24d76 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:55.187027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac648c6-19e5-4743-8747-6d74ecc24d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:55.187303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac648c6-19e5-4743-8747-6d74ecc24d76 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:55.188037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac648c6-19e5-4743-8747-6d74ecc24d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:55.188343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac648c6-19e5-4743-8747-6d74ecc24d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:55.189759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac648c6-19e5-4743-8747-6d74ecc24d76 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:55.220039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 308/616] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:55.222075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:55.225256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac648c6-19e5-4743-8747-6d74ecc24d76 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:55.249830 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e0ade-0fe2-4a7a-9496-dfff850c3119 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:55.287823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306955e-ceed-4afd-91f1-eb1bbd86eb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:55.289370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306955e-ceed-4afd-91f1-eb1bbd86eb83 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:55.289900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306955e-ceed-4afd-91f1-eb1bbd86eb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:55.290624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306955e-ceed-4afd-91f1-eb1bbd86eb83 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:55.291619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306955e-ceed-4afd-91f1-eb1bbd86eb83 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:55.292144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306955e-ceed-4afd-91f1-eb1bbd86eb83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:55.294776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306955e-ceed-4afd-91f1-eb1bbd86eb83 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:55.309575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 309/617] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:55.311318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306955e-ceed-4afd-91f1-eb1bbd86eb83 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:55.425037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306955e-ceed-4afd-91f1-eb1bbd86eb83 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:55.459526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:55.466408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cd4ed-faa5-4b79-bf54-76a745db2fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:55.467446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cd4ed-faa5-4b79-bf54-76a745db2fcc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:55.468029 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cd4ed-faa5-4b79-bf54-76a745db2fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:55.468415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cd4ed-faa5-4b79-bf54-76a745db2fcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:55.481203 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cd4ed-faa5-4b79-bf54-76a745db2fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:55.483253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cd4ed-faa5-4b79-bf54-76a745db2fcc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c74cd4ed-faa5-4b79-bf54-76a745db2fcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:55.502209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:55.503296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:55.504404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:55.504404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:55.504404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:55.504404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:55.510188 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 310/618] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:55.511710 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cd4ed-faa5-4b79-bf54-76a745db2fcc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:55.552987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac648c6-19e5-4743-8747-6d74ecc24d76 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:55.626873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cd4ed-faa5-4b79-bf54-76a745db2fcc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:55.666941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b7af5-da5e-461a-81f9-5a30ecd2865b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:55.667972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7af5-da5e-461a-81f9-5a30ecd2865b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:55.668966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7af5-da5e-461a-81f9-5a30ecd2865b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:55.669354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7af5-da5e-461a-81f9-5a30ecd2865b None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:55.670228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b7af5-da5e-461a-81f9-5a30ecd2865b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:55.671043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b7af5-da5e-461a-81f9-5a30ecd2865b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:55.672612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b7af5-da5e-461a-81f9-5a30ecd2865b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:55.691003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 311/619] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:55.692266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b7af5-da5e-461a-81f9-5a30ecd2865b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:55.706464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834e082-4258-4f79-b790-a6264f973dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:55.707282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834e082-4258-4f79-b790-a6264f973dfc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:55.707565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834e082-4258-4f79-b790-a6264f973dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:55.707845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834e082-4258-4f79-b790-a6264f973dfc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:55.738697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834e082-4258-4f79-b790-a6264f973dfc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:55.740104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834e082-4258-4f79-b790-a6264f973dfc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:55.749202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7834e082-4258-4f79-b790-a6264f973dfc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:55.778207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:55.779281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:55.780234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:55.780234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:55.780234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:55.780234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:55.780234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:55.780234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:55.780234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:55.780234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:55.780234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 16:06:55.780234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:55.780234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 309/620] 10.222.0.42 () {56 vars in 1093 bytes} [Wed May 25 16:06:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 555 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:06:55.785435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834e082-4258-4f79-b790-a6264f973dfc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:55.807021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b7af5-da5e-461a-81f9-5a30ecd2865b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:55.809336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:55.858841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e61e8-0f06-4a5b-bd36-bda72c94e4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:55.859759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e61e8-0f06-4a5b-bd36-bda72c94e4ad None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:55.860521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e61e8-0f06-4a5b-bd36-bda72c94e4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:55.860803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e61e8-0f06-4a5b-bd36-bda72c94e4ad None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:55.869027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e61e8-0f06-4a5b-bd36-bda72c94e4ad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:55.870694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e61e8-0f06-4a5b-bd36-bda72c94e4ad None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-915e61e8-0f06-4a5b-bd36-bda72c94e4ad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:55.888291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:55.889891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:55.891446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:55.891446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:55.891446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:55.891446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:55.891446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:55.891446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:55.891446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:55.891446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:55.891446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 16:06:55.891446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:55.894059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 312/621] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:06:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:55.895931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e61e8-0f06-4a5b-bd36-bda72c94e4ad None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:56.006349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e61e8-0f06-4a5b-bd36-bda72c94e4ad None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:56.015220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:56.045285 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b7024-e9e8-458f-a36e-0ed2e3fd7e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:56.046010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b7024-e9e8-458f-a36e-0ed2e3fd7e17 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:56.046407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b7024-e9e8-458f-a36e-0ed2e3fd7e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:56.046694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b7024-e9e8-458f-a36e-0ed2e3fd7e17 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:56.047645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b7024-e9e8-458f-a36e-0ed2e3fd7e17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:56.047963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b7024-e9e8-458f-a36e-0ed2e3fd7e17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:56.049618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b7024-e9e8-458f-a36e-0ed2e3fd7e17 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:56.066697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834e082-4258-4f79-b790-a6264f973dfc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:56.075025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 313/622] 10.222.0.42 () {56 vars in 1135 bytes} [Wed May 25 16:06:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:06:56.076484 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b7024-e9e8-458f-a36e-0ed2e3fd7e17 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:56.122283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a141fa-266b-4633-8dc2-fd4a45bc6639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:56.123156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a141fa-266b-4633-8dc2-fd4a45bc6639 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:56.123441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a141fa-266b-4633-8dc2-fd4a45bc6639 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:56.123715 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a141fa-266b-4633-8dc2-fd4a45bc6639 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:56.124591 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a141fa-266b-4633-8dc2-fd4a45bc6639 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:56.124899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a141fa-266b-4633-8dc2-fd4a45bc6639 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:56.126445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a141fa-266b-4633-8dc2-fd4a45bc6639 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:56.147152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 310/623] 10.222.0.42 () {56 vars in 1133 bytes} [Wed May 25 16:06:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:56.152340 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a141fa-266b-4633-8dc2-fd4a45bc6639 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:56.296494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:56.306938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b7024-e9e8-458f-a36e-0ed2e3fd7e17 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:56.372132 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a141fa-266b-4633-8dc2-fd4a45bc6639 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:56.410060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c842031-09ab-4411-a725-308cb28d9bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:56.411488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c842031-09ab-4411-a725-308cb28d9bed None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:56.411976 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c842031-09ab-4411-a725-308cb28d9bed None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:56.412149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe65ce6-7361-4012-b120-0284c199d6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:56.412628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c842031-09ab-4411-a725-308cb28d9bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:56.412745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe65ce6-7361-4012-b120-0284c199d6fd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:56.413030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe65ce6-7361-4012-b120-0284c199d6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:56.413314 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe65ce6-7361-4012-b120-0284c199d6fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:56.446627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c842031-09ab-4411-a725-308cb28d9bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:56.448237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c842031-09ab-4411-a725-308cb28d9bed None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:56.493098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe65ce6-7361-4012-b120-0284c199d6fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4c842031-09ab-4411-a725-308cb28d9bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:56.493851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:56.494775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:56.502765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:56.502765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:56.502765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:56.502765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:56.502765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 314/624] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:56.505457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe65ce6-7361-4012-b120-0284c199d6fd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:56.506538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477d6cc-b265-411c-bfa9-33f130d2aa4b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:56.506824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477d6cc-b265-411c-bfa9-33f130d2aa4b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:56.507088 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477d6cc-b265-411c-bfa9-33f130d2aa4b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:56.508808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 315/625] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:06:56.513651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477d6cc-b265-411c-bfa9-33f130d2aa4b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:56.534401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:56.592855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:56.642819 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477d6cc-b265-411c-bfa9-33f130d2aa4b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:56.644414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 311/626] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:56.646233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f42612-5d33-4e48-8bd9-a3b1ea676cf7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:56.647326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f42612-5d33-4e48-8bd9-a3b1ea676cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:56.647611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f42612-5d33-4e48-8bd9-a3b1ea676cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:56.670201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-00f42612-5d33-4e48-8bd9-a3b1ea676cf7 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:56.672252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-00f42612-5d33-4e48-8bd9-a3b1ea676cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:56.683098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c09fc-73ba-439d-b8b5-c19fbc9748c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:56.683856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c09fc-73ba-439d-b8b5-c19fbc9748c2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:56.684679 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c09fc-73ba-439d-b8b5-c19fbc9748c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:56.684959 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c09fc-73ba-439d-b8b5-c19fbc9748c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:56.685707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c09fc-73ba-439d-b8b5-c19fbc9748c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:56.686042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c09fc-73ba-439d-b8b5-c19fbc9748c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:56.687599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c09fc-73ba-439d-b8b5-c19fbc9748c2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:56.706465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 316/627] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:56.706465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c09fc-73ba-439d-b8b5-c19fbc9748c2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:56.782244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 312/628] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:56.783465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f42612-5d33-4e48-8bd9-a3b1ea676cf7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:56.810040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:56.847169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c09fc-73ba-439d-b8b5-c19fbc9748c2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:56.893075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7491be-f31c-44c1-83e4-207185cbc07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:56.893929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7491be-f31c-44c1-83e4-207185cbc07e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:56.894218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7491be-f31c-44c1-83e4-207185cbc07e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:56.894487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7491be-f31c-44c1-83e4-207185cbc07e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:56.897693 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:56.907325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7491be-f31c-44c1-83e4-207185cbc07e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:56.907532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f42612-5d33-4e48-8bd9-a3b1ea676cf7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:56.908639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7491be-f31c-44c1-83e4-207185cbc07e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-dc7491be-f31c-44c1-83e4-207185cbc07e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:56.930510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:56.931462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:56.932338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:56.932338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:56.932338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:56.932338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:56.932558 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 317/629] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:56.933365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7491be-f31c-44c1-83e4-207185cbc07e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:56.956410 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c46cd-2ca0-4eb2-a5e2-23121ae81058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:56.957221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c46cd-2ca0-4eb2-a5e2-23121ae81058 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:56.957514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c46cd-2ca0-4eb2-a5e2-23121ae81058 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:56.958597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c46cd-2ca0-4eb2-a5e2-23121ae81058 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:56.967795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c46cd-2ca0-4eb2-a5e2-23121ae81058 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:56.969682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c46cd-2ca0-4eb2-a5e2-23121ae81058 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c10c46cd-2ca0-4eb2-a5e2-23121ae81058 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:06:56.987107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:56.988074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:56.988966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:06:56.988966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:06:56.988966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:06:56.988966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:06:56.988966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:06:56.988966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:06:56.988966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:56.988966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:56.992955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 313/630] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:56.994417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0f06b-55ea-4887-be9c-c8df28758371 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:56.994706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0f06b-55ea-4887-be9c-c8df28758371 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:56.994986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0f06b-55ea-4887-be9c-c8df28758371 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:56.996795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 314/631] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:56.997503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0f06b-55ea-4887-be9c-c8df28758371 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:57.062342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7491be-f31c-44c1-83e4-207185cbc07e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:57.092554 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:57.102724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c96ed-a56e-46d8-9ddd-46aea5d457df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:57.103760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c96ed-a56e-46d8-9ddd-46aea5d457df None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.104226 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c96ed-a56e-46d8-9ddd-46aea5d457df None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.104698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c96ed-a56e-46d8-9ddd-46aea5d457df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.106106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0f06b-55ea-4887-be9c-c8df28758371 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:57.116119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c96ed-a56e-46d8-9ddd-46aea5d457df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c02c96ed-a56e-46d8-9ddd-46aea5d457df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:57.125211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:57.126220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:57.127035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:57.127675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 318/632] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:57.128165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c96ed-a56e-46d8-9ddd-46aea5d457df None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:57.150756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2aa031-f1c2-4f32-85d0-4dc835bd1abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:57.151452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aa031-f1c2-4f32-85d0-4dc835bd1abe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.151738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aa031-f1c2-4f32-85d0-4dc835bd1abe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.152017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aa031-f1c2-4f32-85d0-4dc835bd1abe None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.152773 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2aa031-f1c2-4f32-85d0-4dc835bd1abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:57.153086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2aa031-f1c2-4f32-85d0-4dc835bd1abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:57.154765 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2aa031-f1c2-4f32-85d0-4dc835bd1abe None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:57.173733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 315/633] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:57.173733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13307513-62d7-493e-998e-ee2fd3a7488f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.173733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13307513-62d7-493e-998e-ee2fd3a7488f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.173733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13307513-62d7-493e-998e-ee2fd3a7488f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.194035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-13307513-62d7-493e-998e-ee2fd3a7488f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:57.195231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-13307513-62d7-493e-998e-ee2fd3a7488f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:57.207255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:57.247388 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c96ed-a56e-46d8-9ddd-46aea5d457df None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:57.276022 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 316/634] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:57.276641 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13307513-62d7-493e-998e-ee2fd3a7488f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:57.298845 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3e00e-9771-4312-9165-0d5bc4ba65c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:57.300427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3e00e-9771-4312-9165-0d5bc4ba65c4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.300901 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3e00e-9771-4312-9165-0d5bc4ba65c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.301605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3e00e-9771-4312-9165-0d5bc4ba65c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.302839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3e00e-9771-4312-9165-0d5bc4ba65c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:57.303159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3e00e-9771-4312-9165-0d5bc4ba65c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:57.305067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3e00e-9771-4312-9165-0d5bc4ba65c4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:57.322846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 319/635] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:06:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:57.322846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3e00e-9771-4312-9165-0d5bc4ba65c4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:57.371202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:57.391673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13307513-62d7-493e-998e-ee2fd3a7488f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:57.432284 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6decf72-1d8e-4530-b1cd-f751ab75c797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:57.433245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6decf72-1d8e-4530-b1cd-f751ab75c797 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6decf72-1d8e-4530-b1cd-f751ab75c797 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.433879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6decf72-1d8e-4530-b1cd-f751ab75c797 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.435448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3e00e-9771-4312-9165-0d5bc4ba65c4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:57.435589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6decf72-1d8e-4530-b1cd-f751ab75c797 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:57.435896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6decf72-1d8e-4530-b1cd-f751ab75c797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:57.437279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6decf72-1d8e-4530-b1cd-f751ab75c797 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:57.445655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 317/636] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:57.446501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6decf72-1d8e-4530-b1cd-f751ab75c797 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:57.457491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:57.481305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b917c-818b-4203-8647-7e00debfa1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:57.482224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b917c-818b-4203-8647-7e00debfa1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.482510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b917c-818b-4203-8647-7e00debfa1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.482788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b917c-818b-4203-8647-7e00debfa1e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.489614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b917c-818b-4203-8647-7e00debfa1e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:57.491084 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b917c-818b-4203-8647-7e00debfa1e9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6a8b917c-818b-4203-8647-7e00debfa1e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:06:57.505657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:57.506657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:57.507559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:06:57.507559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:06:57.507559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:06:57.507559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:06:57.507559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:06:57.507559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:06:57.507559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:06:57.507559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:57.507559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 320/637] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:06:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:57.508049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b917c-818b-4203-8647-7e00debfa1e9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:57.558937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6decf72-1d8e-4530-b1cd-f751ab75c797 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:57.599907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41bc99d-aa99-424e-a0e8-589c86fca5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:57.600708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bc99d-aa99-424e-a0e8-589c86fca5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.600993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bc99d-aa99-424e-a0e8-589c86fca5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.601270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bc99d-aa99-424e-a0e8-589c86fca5c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.612231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41bc99d-aa99-424e-a0e8-589c86fca5c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c41bc99d-aa99-424e-a0e8-589c86fca5c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:57.619992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:57.621004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:57.622601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 318/638] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:57.623335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9378ed-23fa-4e5b-a72f-12302959b39a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.623335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9378ed-23fa-4e5b-a72f-12302959b39a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.623712 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9378ed-23fa-4e5b-a72f-12302959b39a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.624803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 319/639] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:57.625502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9378ed-23fa-4e5b-a72f-12302959b39a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:57.635301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b917c-818b-4203-8647-7e00debfa1e9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:57.642777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:57.670540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa182ad-7518-4c58-9d31-13292ec08b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:57.671431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa182ad-7518-4c58-9d31-13292ec08b57 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.671710 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa182ad-7518-4c58-9d31-13292ec08b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.671980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa182ad-7518-4c58-9d31-13292ec08b57 None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/c6eb13611d0d4b6abe035c9bfbfde79c/roles/6bc01dbf83c844fe802466d3c4ada6af` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.687197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa182ad-7518-4c58-9d31-13292ec08b57 None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=c6eb13611d0d4b6abe035c9bfbfde79c, role_id=6bc01dbf83c844fe802466d3c4ada6af)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:57.688637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa182ad-7518-4c58-9d31-13292ec08b57 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:57.692658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:57.702786 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 321/640] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:06:57 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/c6eb13611d0d4b6abe035c9bfbfde79c/roles/6bc01dbf83c844fe802466d3c4ada6af => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:06:57.703604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa182ad-7518-4c58-9d31-13292ec08b57 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:57.732658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9378ed-23fa-4e5b-a72f-12302959b39a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:57.772322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2551d5-e457-44ef-af80-53edbf431f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:57.774188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2551d5-e457-44ef-af80-53edbf431f13 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.774817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2551d5-e457-44ef-af80-53edbf431f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.775291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2551d5-e457-44ef-af80-53edbf431f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.791621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2551d5-e457-44ef-af80-53edbf431f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:57.794214 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2551d5-e457-44ef-af80-53edbf431f13 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3f2551d5-e457-44ef-af80-53edbf431f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:57.814759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:57.815909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:57.816815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:57.816815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:57.816815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:57.816815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:57.816815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 320/641] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:57.817142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2551d5-e457-44ef-af80-53edbf431f13 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:57.838702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa182ad-7518-4c58-9d31-13292ec08b57 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:57.876469 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07c0bc-b926-456b-baf0-b886464e6fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:57.877070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c0bc-b926-456b-baf0-b886464e6fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.877356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c0bc-b926-456b-baf0-b886464e6fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.877635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c0bc-b926-456b-baf0-b886464e6fe4 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.878471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07c0bc-b926-456b-baf0-b886464e6fe4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:57.878785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07c0bc-b926-456b-baf0-b886464e6fe4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:57.880243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07c0bc-b926-456b-baf0-b886464e6fe4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:57.896894 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 322/642] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:06:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:57.898179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9eee9-f826-41f7-ac31-af46fa6bff2a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.898503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9eee9-f826-41f7-ac31-af46fa6bff2a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.898762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9eee9-f826-41f7-ac31-af46fa6bff2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.919811 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2ff9eee9-f826-41f7-ac31-af46fa6bff2a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:57.921038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2ff9eee9-f826-41f7-ac31-af46fa6bff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:57.932726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:57.940690 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2551d5-e457-44ef-af80-53edbf431f13 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:57.960108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:57.978068 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f6aab-8032-43db-b99e-309f8729d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:57.978747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f6aab-8032-43db-b99e-309f8729d843 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:57.979034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f6aab-8032-43db-b99e-309f8729d843 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:57.979316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f6aab-8032-43db-b99e-309f8729d843 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:57.980070 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f6aab-8032-43db-b99e-309f8729d843 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:57.980381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f6aab-8032-43db-b99e-309f8729d843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:57.981997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f6aab-8032-43db-b99e-309f8729d843 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:57.990512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 321/643] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:57.991146 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f6aab-8032-43db-b99e-309f8729d843 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:58.008272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 323/644] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:06:58.009404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9eee9-f826-41f7-ac31-af46fa6bff2a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:58.116425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9eee9-f826-41f7-ac31-af46fa6bff2a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:58.119621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f6aab-8032-43db-b99e-309f8729d843 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:58.153585 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17766e50-562a-4ee5-80ea-49792d87086f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:58.154199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17766e50-562a-4ee5-80ea-49792d87086f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:58.154481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17766e50-562a-4ee5-80ea-49792d87086f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:58.154758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17766e50-562a-4ee5-80ea-49792d87086f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:58.155635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17766e50-562a-4ee5-80ea-49792d87086f None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=6e6795b5b5cf4499ad494b875756ba71) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:58.155952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17766e50-562a-4ee5-80ea-49792d87086f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:58.156066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8634c-e56d-4661-9c7b-33d1ad74976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:58.156675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8634c-e56d-4661-9c7b-33d1ad74976e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:58.156957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8634c-e56d-4661-9c7b-33d1ad74976e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:58.157230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8634c-e56d-4661-9c7b-33d1ad74976e None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:58.157387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17766e50-562a-4ee5-80ea-49792d87086f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:58.157987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8634c-e56d-4661-9c7b-33d1ad74976e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:58.158347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8634c-e56d-4661-9c7b-33d1ad74976e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:58.160070 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8634c-e56d-4661-9c7b-33d1ad74976e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:58.174044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 324/645] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:06:58 2022] GET /identity/v3/role_assignments?user.id=6e6795b5b5cf4499ad494b875756ba71&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:58.174044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17766e50-562a-4ee5-80ea-49792d87086f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:58.174324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 322/646] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:58.174324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8634c-e56d-4661-9c7b-33d1ad74976e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:58.194817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:58.233347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:58.302776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17766e50-562a-4ee5-80ea-49792d87086f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:58.313613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8634c-e56d-4661-9c7b-33d1ad74976e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:58.353841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e984cab-e45f-43d8-af42-c1074247f605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:58.354400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff5356-3f23-4d25-97b6-775953f76852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:58.354786 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e984cab-e45f-43d8-af42-c1074247f605 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:58.354971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff5356-3f23-4d25-97b6-775953f76852 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:58.355180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e984cab-e45f-43d8-af42-c1074247f605 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:58.355277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff5356-3f23-4d25-97b6-775953f76852 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:58.355513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff5356-3f23-4d25-97b6-775953f76852 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:58.355639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e984cab-e45f-43d8-af42-c1074247f605 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:58.368304 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff5356-3f23-4d25-97b6-775953f76852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:06:58.370824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e984cab-e45f-43d8-af42-c1074247f605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:58.373093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e984cab-e45f-43d8-af42-c1074247f605 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f1ff5356-3f23-4d25-97b6-775953f76852 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:06:58.376604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:58.378589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 323/647] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:58.380530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff5356-3f23-4d25-97b6-775953f76852 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8e984cab-e45f-43d8-af42-c1074247f605 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:58.397594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:58.399569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:58.401943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:58.401943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:58.401943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:58.401943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:58.401943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 325/648] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:58.401943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e984cab-e45f-43d8-af42-c1074247f605 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:58.496199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:58.518927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:58.522644 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff5356-3f23-4d25-97b6-775953f76852 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:58.548769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e984cab-e45f-43d8-af42-c1074247f605 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:58.567143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a61082-1345-4ea9-a509-975bf72b4c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:58.567957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61082-1345-4ea9-a509-975bf72b4c4a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:58.568247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61082-1345-4ea9-a509-975bf72b4c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:58.568526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61082-1345-4ea9-a509-975bf72b4c4a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:58.577736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a61082-1345-4ea9-a509-975bf72b4c4a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:58.579225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a61082-1345-4ea9-a509-975bf72b4c4a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f0a61082-1345-4ea9-a509-975bf72b4c4a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:58.594302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:58.595294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:58.596231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:58.596231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:58.596231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:58.596231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:58.596231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:58.596231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:58.596231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:58.596231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:58.596231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 16:06:58.596231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:58.596231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 324/649] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:58.596840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a61082-1345-4ea9-a509-975bf72b4c4a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:58.598478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713021ad-9127-4a3e-abd3-cd6699ccab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:58.599459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713021ad-9127-4a3e-abd3-cd6699ccab3f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:58.600279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713021ad-9127-4a3e-abd3-cd6699ccab3f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:58.601769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713021ad-9127-4a3e-abd3-cd6699ccab3f None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:58.602765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713021ad-9127-4a3e-abd3-cd6699ccab3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:58.603289 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713021ad-9127-4a3e-abd3-cd6699ccab3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:58.604863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713021ad-9127-4a3e-abd3-cd6699ccab3f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:58.618443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 326/650] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:06:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:58.619604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713021ad-9127-4a3e-abd3-cd6699ccab3f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:58.725682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a61082-1345-4ea9-a509-975bf72b4c4a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:58.737339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713021ad-9127-4a3e-abd3-cd6699ccab3f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:58.761951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:58.765169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8c504-1ddc-47c6-a490-94124ed41ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:58.766104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8c504-1ddc-47c6-a490-94124ed41ccf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:58.766591 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8c504-1ddc-47c6-a490-94124ed41ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:58.767067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8c504-1ddc-47c6-a490-94124ed41ccf None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:58.768205 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8c504-1ddc-47c6-a490-94124ed41ccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:58.768970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8c504-1ddc-47c6-a490-94124ed41ccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:58.770590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8c504-1ddc-47c6-a490-94124ed41ccf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:58.774333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aa494-442a-4277-ad48-1d2eceaa1bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:58.775008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aa494-442a-4277-ad48-1d2eceaa1bfd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:58.775294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aa494-442a-4277-ad48-1d2eceaa1bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:58.775572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aa494-442a-4277-ad48-1d2eceaa1bfd None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:58.776407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4aa494-442a-4277-ad48-1d2eceaa1bfd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:58.776716 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4aa494-442a-4277-ad48-1d2eceaa1bfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:58.778698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4aa494-442a-4277-ad48-1d2eceaa1bfd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:58.786820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 325/651] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:58.788008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8c504-1ddc-47c6-a490-94124ed41ccf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:58.800128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 327/652] 10.222.0.42 () {56 vars in 1137 bytes} [Wed May 25 16:06:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:58.801206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aa494-442a-4277-ad48-1d2eceaa1bfd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:58.812379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:58.898504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8c504-1ddc-47c6-a490-94124ed41ccf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:58.911701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aa494-442a-4277-ad48-1d2eceaa1bfd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:58.936387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19187192-6c0b-4990-a5e0-402e60dd5882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:58.937534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19187192-6c0b-4990-a5e0-402e60dd5882 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:58.938037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19187192-6c0b-4990-a5e0-402e60dd5882 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:58.938541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19187192-6c0b-4990-a5e0-402e60dd5882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:58.950060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa02085-e4a8-4254-9d9b-7341dd411ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:58.950872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa02085-e4a8-4254-9d9b-7341dd411ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:58.951157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa02085-e4a8-4254-9d9b-7341dd411ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:58.951358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19187192-6c0b-4990-a5e0-402e60dd5882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:58.951480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa02085-e4a8-4254-9d9b-7341dd411ed4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:58.953329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19187192-6c0b-4990-a5e0-402e60dd5882 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:58.959285 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa02085-e4a8-4254-9d9b-7341dd411ed4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:58.960799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa02085-e4a8-4254-9d9b-7341dd411ed4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-19187192-6c0b-4990-a5e0-402e60dd5882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:58.972981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:58.974416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:58.976363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:58.976363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:58.976363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:58.976363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9aa02085-e4a8-4254-9d9b-7341dd411ed4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:58.977238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:58.979968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 328/653] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:06:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:58.983354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa02085-e4a8-4254-9d9b-7341dd411ed4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:58.986660 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 326/654] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:58.987842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19187192-6c0b-4990-a5e0-402e60dd5882 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:59.033482 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:59.069402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:59.098849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa02085-e4a8-4254-9d9b-7341dd411ed4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:59.103795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19187192-6c0b-4990-a5e0-402e60dd5882 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:59.135784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894588f-609b-48cd-a0cd-41b16b55110b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:59.136301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894588f-609b-48cd-a0cd-41b16b55110b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:59.136589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894588f-609b-48cd-a0cd-41b16b55110b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:59.137392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894588f-609b-48cd-a0cd-41b16b55110b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:59.142228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deab4ec-dd4f-48ef-b6e8-53f67e8e8eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:59.143183 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deab4ec-dd4f-48ef-b6e8-53f67e8e8eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:59.143750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deab4ec-dd4f-48ef-b6e8-53f67e8e8eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:59.144274 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deab4ec-dd4f-48ef-b6e8-53f67e8e8eb9 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:59.145347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deab4ec-dd4f-48ef-b6e8-53f67e8e8eb9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:59.145892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deab4ec-dd4f-48ef-b6e8-53f67e8e8eb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:59.147709 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deab4ec-dd4f-48ef-b6e8-53f67e8e8eb9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:59.167761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 327/655] 10.222.0.42 () {56 vars in 1137 bytes} [Wed May 25 16:06:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:59.169420 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deab4ec-dd4f-48ef-b6e8-53f67e8e8eb9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:59.217350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894588f-609b-48cd-a0cd-41b16b55110b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:59.218639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894588f-609b-48cd-a0cd-41b16b55110b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:59.277281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:59.285066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deab4ec-dd4f-48ef-b6e8-53f67e8e8eb9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:59.319832 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:59.326371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070258-18d8-4724-8e44-060aba17d430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:59.327043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070258-18d8-4724-8e44-060aba17d430 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:59.327326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070258-18d8-4724-8e44-060aba17d430 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:59.327599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070258-18d8-4724-8e44-060aba17d430 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:59.328326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63070258-18d8-4724-8e44-060aba17d430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:59.328631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63070258-18d8-4724-8e44-060aba17d430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:59.330044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63070258-18d8-4724-8e44-060aba17d430 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:59.342891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 328/656] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:59.343772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070258-18d8-4724-8e44-060aba17d430 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:59.364809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 329/657] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:59.366984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894588f-609b-48cd-a0cd-41b16b55110b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:59.462085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070258-18d8-4724-8e44-060aba17d430 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:59.503337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1cb95-9c9c-4342-85b0-db5c782cc2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:59.504540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1cb95-9c9c-4342-85b0-db5c782cc2cb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:59.504916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1cb95-9c9c-4342-85b0-db5c782cc2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:59.505311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1cb95-9c9c-4342-85b0-db5c782cc2cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:59.514659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894588f-609b-48cd-a0cd-41b16b55110b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:59.555969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7defa37-b6f7-4578-a0a5-ac3038542653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:59.556834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7defa37-b6f7-4578-a0a5-ac3038542653 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:59.557160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7defa37-b6f7-4578-a0a5-ac3038542653 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:59.557503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7defa37-b6f7-4578-a0a5-ac3038542653 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:59.568126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7defa37-b6f7-4578-a0a5-ac3038542653 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:59.570214 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7defa37-b6f7-4578-a0a5-ac3038542653 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:59.576550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:59.590180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1cb95-9c9c-4342-85b0-db5c782cc2cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d7defa37-b6f7-4578-a0a5-ac3038542653 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:06:59.592069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:59.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:59.594253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:59.594253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:59.594253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:06:59.594253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:06:59.594253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:06:59.594253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:06:59.594253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:06:59.594253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:06:59.594253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 16:06:59.594253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:59.594824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1cb95-9c9c-4342-85b0-db5c782cc2cb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:59.598290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 330/658] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:06:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:59.600101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7defa37-b6f7-4578-a0a5-ac3038542653 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:59.602984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:59.718566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 329/659] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:06:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:06:59.719889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1cb95-9c9c-4342-85b0-db5c782cc2cb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:59.733808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7defa37-b6f7-4578-a0a5-ac3038542653 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:59.787917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049198f6-3404-4b71-b03f-3258327fd53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:59.789128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049198f6-3404-4b71-b03f-3258327fd53c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:59.789426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049198f6-3404-4b71-b03f-3258327fd53c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:59.789823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049198f6-3404-4b71-b03f-3258327fd53c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:59.805652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049198f6-3404-4b71-b03f-3258327fd53c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:59.807283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049198f6-3404-4b71-b03f-3258327fd53c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:59.827830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-049198f6-3404-4b71-b03f-3258327fd53c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:06:59.835049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:06:59.836156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:06:59.837445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:06:59.837445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:06:59.837445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:06:59.837445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:06:59.837445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 331/660] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:06:59.837445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049198f6-3404-4b71-b03f-3258327fd53c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:59.843143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1cb95-9c9c-4342-85b0-db5c782cc2cb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:59.879832 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0dde5-a917-4b5c-9426-f7f1eea5558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:59.880818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0dde5-a917-4b5c-9426-f7f1eea5558b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:59.881292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0dde5-a917-4b5c-9426-f7f1eea5558b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:59.881780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0dde5-a917-4b5c-9426-f7f1eea5558b None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:59.882962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0dde5-a917-4b5c-9426-f7f1eea5558b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:06:59.883445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0dde5-a917-4b5c-9426-f7f1eea5558b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:59.885167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0dde5-a917-4b5c-9426-f7f1eea5558b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:59.914408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 330/661] 10.222.0.42 () {56 vars in 1139 bytes} [Wed May 25 16:06:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:06:59.914408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0dde5-a917-4b5c-9426-f7f1eea5558b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:59.916900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:59.969379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049198f6-3404-4b71-b03f-3258327fd53c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:00.012249 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb89d74-e64a-4cdf-9793-97c42a87aee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:00.013739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb89d74-e64a-4cdf-9793-97c42a87aee2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.014225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb89d74-e64a-4cdf-9793-97c42a87aee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.014694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb89d74-e64a-4cdf-9793-97c42a87aee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.025932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb89d74-e64a-4cdf-9793-97c42a87aee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:00.028296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb89d74-e64a-4cdf-9793-97c42a87aee2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-bdb89d74-e64a-4cdf-9793-97c42a87aee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:00.048320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:00.049233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:00.050158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:00.050158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:00.050158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:00.050158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:00.052160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 332/662] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:06:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:00.053355 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0dde5-a917-4b5c-9426-f7f1eea5558b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:00.054391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb89d74-e64a-4cdf-9793-97c42a87aee2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:00.099830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d681ef-465a-4631-a22f-a91e8f1be8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:00.100410 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d681ef-465a-4631-a22f-a91e8f1be8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.100696 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d681ef-465a-4631-a22f-a91e8f1be8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.100977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d681ef-465a-4631-a22f-a91e8f1be8f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.101708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d681ef-465a-4631-a22f-a91e8f1be8f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:00.102038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d681ef-465a-4631-a22f-a91e8f1be8f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:00.103419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d681ef-465a-4631-a22f-a91e8f1be8f9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:00.121768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 331/663] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:00.121768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d681ef-465a-4631-a22f-a91e8f1be8f9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:00.132025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:00.175759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb89d74-e64a-4cdf-9793-97c42a87aee2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:00.187159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:00.228137 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439cb4c-6b03-4711-909a-1a6f6d68ad7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:00.228629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d681ef-465a-4631-a22f-a91e8f1be8f9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:00.229085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439cb4c-6b03-4711-909a-1a6f6d68ad7c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.229085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439cb4c-6b03-4711-909a-1a6f6d68ad7c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.229337 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439cb4c-6b03-4711-909a-1a6f6d68ad7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.265757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a81288-89b5-4c74-8ca3-5d14d1d03078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:00.266690 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81288-89b5-4c74-8ca3-5d14d1d03078 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.266972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81288-89b5-4c74-8ca3-5d14d1d03078 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.267242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81288-89b5-4c74-8ca3-5d14d1d03078 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.267962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a81288-89b5-4c74-8ca3-5d14d1d03078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:00.268265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a81288-89b5-4c74-8ca3-5d14d1d03078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:00.270453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a81288-89b5-4c74-8ca3-5d14d1d03078 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:00.283343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 332/664] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:00.284731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731890af-b4b5-4738-9eca-5aff61a5697e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.285017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731890af-b4b5-4738-9eca-5aff61a5697e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.285291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731890af-b4b5-4738-9eca-5aff61a5697e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.289837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 333/665] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:00.289837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731890af-b4b5-4738-9eca-5aff61a5697e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:00.315801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439cb4c-6b03-4711-909a-1a6f6d68ad7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:00.317248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439cb4c-6b03-4711-909a-1a6f6d68ad7c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:00.366044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:00.406390 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731890af-b4b5-4738-9eca-5aff61a5697e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:00.451066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 333/666] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:07:00.451066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439cb4c-6b03-4711-909a-1a6f6d68ad7c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:00.456773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:00.466634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf76649-8a21-4473-96b6-8f02683c6cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:00.467439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf76649-8a21-4473-96b6-8f02683c6cfb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.467885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf76649-8a21-4473-96b6-8f02683c6cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.468170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf76649-8a21-4473-96b6-8f02683c6cfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.475729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf76649-8a21-4473-96b6-8f02683c6cfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:00.477287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf76649-8a21-4473-96b6-8f02683c6cfb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-fcf76649-8a21-4473-96b6-8f02683c6cfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:00.492298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:00.493257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:00.494171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:00.494171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:00.494171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:00.494171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:00.494171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:00.494171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:00.494171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:00.494171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:00.494171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 334/667] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:00.495247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3f8b4-9316-46a0-ba99-306823d91250 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.495618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3f8b4-9316-46a0-ba99-306823d91250 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.495956 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3f8b4-9316-46a0-ba99-306823d91250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.514903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6bd3f8b4-9316-46a0-ba99-306823d91250 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:00.516203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6bd3f8b4-9316-46a0-ba99-306823d91250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:00.562118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439cb4c-6b03-4711-909a-1a6f6d68ad7c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:00.593080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 335/668] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:07:00.594075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3f8b4-9316-46a0-ba99-306823d91250 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:00.606630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75169e5b-e113-46fa-b2fa-a58193fa2ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:00.607699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75169e5b-e113-46fa-b2fa-a58193fa2ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.608166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75169e5b-e113-46fa-b2fa-a58193fa2ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.608643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75169e5b-e113-46fa-b2fa-a58193fa2ca7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.616206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75169e5b-e113-46fa-b2fa-a58193fa2ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:00.617948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75169e5b-e113-46fa-b2fa-a58193fa2ca7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-75169e5b-e113-46fa-b2fa-a58193fa2ca7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:00.632629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:00.633569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:00.634487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:00.634487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:00.634487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:00.634487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:00.634487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:00.634487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:00.634487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:00.634487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:00.634487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 334/669] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:00.635506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75169e5b-e113-46fa-b2fa-a58193fa2ca7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:00.664063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:00.706445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:00.714711 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3f8b4-9316-46a0-ba99-306823d91250 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:00.743806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75169e5b-e113-46fa-b2fa-a58193fa2ca7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:00.749968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e638c-622d-4970-8530-1f0b58b4b325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:00.750774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e638c-622d-4970-8530-1f0b58b4b325 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.751059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e638c-622d-4970-8530-1f0b58b4b325 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.751338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e638c-622d-4970-8530-1f0b58b4b325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.761697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e638c-622d-4970-8530-1f0b58b4b325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:00.763718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e638c-622d-4970-8530-1f0b58b4b325 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:00.779312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece35eef-7795-4f3f-9fa3-e68161e47e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:00.779972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece35eef-7795-4f3f-9fa3-e68161e47e8b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.780256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece35eef-7795-4f3f-9fa3-e68161e47e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.780531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece35eef-7795-4f3f-9fa3-e68161e47e8b None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-809e638c-622d-4970-8530-1f0b58b4b325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:00.780814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:00.781854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:00.782788 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:00.782788 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:00.782788 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:00.782788 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:00.783020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece35eef-7795-4f3f-9fa3-e68161e47e8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:00.783020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece35eef-7795-4f3f-9fa3-e68161e47e8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:00.783145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 336/670] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:00.783213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece35eef-7795-4f3f-9fa3-e68161e47e8b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:00.783281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e638c-622d-4970-8530-1f0b58b4b325 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:00.796839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 335/671] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:00.798180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dc631-e5ec-4a81-939d-f0f301c0e7b1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.798468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dc631-e5ec-4a81-939d-f0f301c0e7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.798743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dc631-e5ec-4a81-939d-f0f301c0e7b1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.799966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 336/672] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:00.800661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dc631-e5ec-4a81-939d-f0f301c0e7b1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:00.885864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e638c-622d-4970-8530-1f0b58b4b325 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:00.897142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:00.902577 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dc631-e5ec-4a81-939d-f0f301c0e7b1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:00.920624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839774a2-a0d6-4b86-8ecb-6592c9641b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:00.921409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839774a2-a0d6-4b86-8ecb-6592c9641b82 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.921689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839774a2-a0d6-4b86-8ecb-6592c9641b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.922002 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839774a2-a0d6-4b86-8ecb-6592c9641b82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.928628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:00.932987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839774a2-a0d6-4b86-8ecb-6592c9641b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:00.938280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1187ac-d6a2-43aa-bf36-d13ad26b7b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:00.938938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1187ac-d6a2-43aa-bf36-d13ad26b7b68 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:00.939221 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1187ac-d6a2-43aa-bf36-d13ad26b7b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:00.939498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1187ac-d6a2-43aa-bf36-d13ad26b7b68 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:00.940275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1187ac-d6a2-43aa-bf36-d13ad26b7b68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:00.940585 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1187ac-d6a2-43aa-bf36-d13ad26b7b68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-839774a2-a0d6-4b86-8ecb-6592c9641b82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:00.941473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:00.942398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:07:00.943244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:00.943869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1187ac-d6a2-43aa-bf36-d13ad26b7b68 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:00.943947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 337/673] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:07:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:00.943947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839774a2-a0d6-4b86-8ecb-6592c9641b82 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:00.957826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 337/674] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:00.958644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1187ac-d6a2-43aa-bf36-d13ad26b7b68 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:01.053584 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839774a2-a0d6-4b86-8ecb-6592c9641b82 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:01.062079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1187ac-d6a2-43aa-bf36-d13ad26b7b68 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:01.086784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5cdf0-e0fb-4b7e-b1a5-f2930c12c3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:01.087454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5cdf0-e0fb-4b7e-b1a5-f2930c12c3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:01.087737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5cdf0-e0fb-4b7e-b1a5-f2930c12c3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:01.088014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5cdf0-e0fb-4b7e-b1a5-f2930c12c3d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:01.088766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5cdf0-e0fb-4b7e-b1a5-f2930c12c3d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:01.089068 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5cdf0-e0fb-4b7e-b1a5-f2930c12c3d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:01.091148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5cdf0-e0fb-4b7e-b1a5-f2930c12c3d3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:01.094536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14117e43-8ce5-4697-868f-b0b88c179976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:01.095187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14117e43-8ce5-4697-868f-b0b88c179976 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:01.095466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14117e43-8ce5-4697-868f-b0b88c179976 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:01.095737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14117e43-8ce5-4697-868f-b0b88c179976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:01.096546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14117e43-8ce5-4697-868f-b0b88c179976 None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=b9913b9eb502434985f84c4a11632ccc) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:01.096854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14117e43-8ce5-4697-868f-b0b88c179976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:01.098288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14117e43-8ce5-4697-868f-b0b88c179976 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:01.102853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 338/675] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:01.104028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 338/676] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:00 2022] GET /identity/v3/role_assignments?user.id=b9913b9eb502434985f84c4a11632ccc&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:01.104466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc11e38-8b64-4d25-a932-7df00087327a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:01.104817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14117e43-8ce5-4697-868f-b0b88c179976 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:01.105032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc11e38-8b64-4d25-a932-7df00087327a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:01.105695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc11e38-8b64-4d25-a932-7df00087327a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:01.130792 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-adc11e38-8b64-4d25-a932-7df00087327a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:01.132030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-adc11e38-8b64-4d25-a932-7df00087327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:01.162939 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:01.173496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:01.218760 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 339/677] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:07:01.219925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc11e38-8b64-4d25-a932-7df00087327a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:01.240615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14117e43-8ce5-4697-868f-b0b88c179976 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:01.284869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ab024-7cdb-43bf-9541-4e3b5c8fb2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:01.285653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ab024-7cdb-43bf-9541-4e3b5c8fb2ff None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:01.286044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ab024-7cdb-43bf-9541-4e3b5c8fb2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:01.286416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ab024-7cdb-43bf-9541-4e3b5c8fb2ff None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:01.287233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ab024-7cdb-43bf-9541-4e3b5c8fb2ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:01.287573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ab024-7cdb-43bf-9541-4e3b5c8fb2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:01.288916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ab024-7cdb-43bf-9541-4e3b5c8fb2ff None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:01.299465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 339/678] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:07:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:01.300311 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ab024-7cdb-43bf-9541-4e3b5c8fb2ff None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:01.360155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc11e38-8b64-4d25-a932-7df00087327a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:01.397508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14781405-4349-40a5-80ff-9338e96172a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:01.398202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14781405-4349-40a5-80ff-9338e96172a4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:01.398489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14781405-4349-40a5-80ff-9338e96172a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:01.398776 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14781405-4349-40a5-80ff-9338e96172a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:01.399617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14781405-4349-40a5-80ff-9338e96172a4 None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=8386ca93fe48460ba5b6f0c10dc51122) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:01.399934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14781405-4349-40a5-80ff-9338e96172a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:01.401387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14781405-4349-40a5-80ff-9338e96172a4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:01.407574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 340/679] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:01 2022] GET /identity/v3/role_assignments?user.id=8386ca93fe48460ba5b6f0c10dc51122&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:01.408399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14781405-4349-40a5-80ff-9338e96172a4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:01.449674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ab024-7cdb-43bf-9541-4e3b5c8fb2ff None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:01.456573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:01.467692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:01.504932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acdf16-9dd7-4347-ad84-0d432db9b76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:01.505804 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acdf16-9dd7-4347-ad84-0d432db9b76b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:01.506098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acdf16-9dd7-4347-ad84-0d432db9b76b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:01.506378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acdf16-9dd7-4347-ad84-0d432db9b76b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:01.514918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acdf16-9dd7-4347-ad84-0d432db9b76b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:01.516154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acdf16-9dd7-4347-ad84-0d432db9b76b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-77acdf16-9dd7-4347-ad84-0d432db9b76b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:01.532799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:01.534049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:01.534961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:01.534961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:01.534961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:01.534961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:01.534961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:01.534961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:01.534961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:01.534961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:01.539432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 340/680] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:01.540677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acdf16-9dd7-4347-ad84-0d432db9b76b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:01.570108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14781405-4349-40a5-80ff-9338e96172a4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:01.625047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbcb71-dba0-4531-aff0-2369e1b3bb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:01.626000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbcb71-dba0-4531-aff0-2369e1b3bb4c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:01.626359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbcb71-dba0-4531-aff0-2369e1b3bb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:01.626708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbcb71-dba0-4531-aff0-2369e1b3bb4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:01.641382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbcb71-dba0-4531-aff0-2369e1b3bb4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-02fbcb71-dba0-4531-aff0-2369e1b3bb4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:01.651764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:01.653301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 341/681] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:01.654986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbcb71-dba0-4531-aff0-2369e1b3bb4c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:01.673680 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acdf16-9dd7-4347-ad84-0d432db9b76b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:01.718890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85751d2-9305-4b6a-8899-dd9fb8cf2436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:01.720395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85751d2-9305-4b6a-8899-dd9fb8cf2436 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:01.720681 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85751d2-9305-4b6a-8899-dd9fb8cf2436 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:01.720963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85751d2-9305-4b6a-8899-dd9fb8cf2436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:01.739851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85751d2-9305-4b6a-8899-dd9fb8cf2436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:01.741368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85751d2-9305-4b6a-8899-dd9fb8cf2436 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d85751d2-9305-4b6a-8899-dd9fb8cf2436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:01.767310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:01.768284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:01.769165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:01.769165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:01.769165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:01.769165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:01.769384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:01.771636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 341/682] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:01.772681 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85751d2-9305-4b6a-8899-dd9fb8cf2436 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:01.791107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbcb71-dba0-4531-aff0-2369e1b3bb4c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:01.804872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:01.830952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a92cae-c97d-4959-be3e-919d4581deac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:01.831629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a92cae-c97d-4959-be3e-919d4581deac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:01.831917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a92cae-c97d-4959-be3e-919d4581deac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:01.832199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a92cae-c97d-4959-be3e-919d4581deac None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:01.833123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a92cae-c97d-4959-be3e-919d4581deac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:01.833440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a92cae-c97d-4959-be3e-919d4581deac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:01.835025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a92cae-c97d-4959-be3e-919d4581deac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:01.854044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 342/683] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:01.854971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a92cae-c97d-4959-be3e-919d4581deac None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:01.892703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85751d2-9305-4b6a-8899-dd9fb8cf2436 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:01.945443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f5893-c19b-436f-992d-05cd8a4592e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:01.947017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f5893-c19b-436f-992d-05cd8a4592e9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:01.947501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f5893-c19b-436f-992d-05cd8a4592e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:01.948379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f5893-c19b-436f-992d-05cd8a4592e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:01.950329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f5893-c19b-436f-992d-05cd8a4592e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:01.950953 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f5893-c19b-436f-992d-05cd8a4592e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:01.953319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f5893-c19b-436f-992d-05cd8a4592e9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:01.973303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 342/684] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:07:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:01.973303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f5893-c19b-436f-992d-05cd8a4592e9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:01.987145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a92cae-c97d-4959-be3e-919d4581deac None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:02.038814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e179f-8e6c-46ef-b5c6-890480eeb42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:02.040134 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e179f-8e6c-46ef-b5c6-890480eeb42c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:02.040597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e179f-8e6c-46ef-b5c6-890480eeb42c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:02.041289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e179f-8e6c-46ef-b5c6-890480eeb42c None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:02.042919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e179f-8e6c-46ef-b5c6-890480eeb42c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:02.043925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e179f-8e6c-46ef-b5c6-890480eeb42c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:02.045522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e179f-8e6c-46ef-b5c6-890480eeb42c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:02.058639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 343/685] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:02.060669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e179f-8e6c-46ef-b5c6-890480eeb42c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:02.072295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:02.076765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:02.097899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f5893-c19b-436f-992d-05cd8a4592e9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:02.147604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda5561-f9fc-4915-9988-de17ddb1c7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:02.148291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda5561-f9fc-4915-9988-de17ddb1c7ce None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:02.148578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda5561-f9fc-4915-9988-de17ddb1c7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:02.148860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda5561-f9fc-4915-9988-de17ddb1c7ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:02.150208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda5561-f9fc-4915-9988-de17ddb1c7ce None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=e42c796dbd7441289a7cbc328abeaf81) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:02.150532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda5561-f9fc-4915-9988-de17ddb1c7ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:02.152725 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda5561-f9fc-4915-9988-de17ddb1c7ce None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:02.161476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 343/686] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:01 2022] GET /identity/v3/role_assignments?user.id=e42c796dbd7441289a7cbc328abeaf81&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:02.162371 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda5561-f9fc-4915-9988-de17ddb1c7ce None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:02.177612 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e179f-8e6c-46ef-b5c6-890480eeb42c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:02.218724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cbf41-686c-458c-8c2f-1bfa8b3ae79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:02.219897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cbf41-686c-458c-8c2f-1bfa8b3ae79d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:02.220387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cbf41-686c-458c-8c2f-1bfa8b3ae79d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:02.220881 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cbf41-686c-458c-8c2f-1bfa8b3ae79d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:02.233739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cbf41-686c-458c-8c2f-1bfa8b3ae79d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:02.235569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cbf41-686c-458c-8c2f-1bfa8b3ae79d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3e8cbf41-686c-458c-8c2f-1bfa8b3ae79d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:02.278176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:02.279244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:02.280177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:02.280177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:02.280177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:02.280177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:02.280177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 344/687] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:02.286603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cbf41-686c-458c-8c2f-1bfa8b3ae79d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:02.295109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda5561-f9fc-4915-9988-de17ddb1c7ce None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:02.335849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ab1ea-85ac-4f47-b639-c4df247f67a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:02.336774 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ab1ea-85ac-4f47-b639-c4df247f67a7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:02.337066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ab1ea-85ac-4f47-b639-c4df247f67a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:02.337343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ab1ea-85ac-4f47-b639-c4df247f67a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:02.346961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ab1ea-85ac-4f47-b639-c4df247f67a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:02.348580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ab1ea-85ac-4f47-b639-c4df247f67a7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-725ab1ea-85ac-4f47-b639-c4df247f67a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:02.364825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:02.365880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:02.366834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:02.366834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:02.366834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:02.366834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:02.366834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:02.366834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:02.366834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:02.366834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:02.366834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 16:07:02.366834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:02.366834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 344/688] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:02.378993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ab1ea-85ac-4f47-b639-c4df247f67a7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:02.385462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:02.395854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:02.449266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cbf41-686c-458c-8c2f-1bfa8b3ae79d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:02.500050 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb736fa-8976-41e7-afcd-d99a077784b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:02.500942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb736fa-8976-41e7-afcd-d99a077784b5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:02.501233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb736fa-8976-41e7-afcd-d99a077784b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:02.501513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb736fa-8976-41e7-afcd-d99a077784b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:02.502924 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb736fa-8976-41e7-afcd-d99a077784b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:02.503239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb736fa-8976-41e7-afcd-d99a077784b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:02.505159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb736fa-8976-41e7-afcd-d99a077784b5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:02.521345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 345/689] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:02.528600 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb736fa-8976-41e7-afcd-d99a077784b5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:02.540982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ab1ea-85ac-4f47-b639-c4df247f67a7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:02.591008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bdd3b-7c01-4f1a-8d6c-24bf24b2e5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:02.591877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bdd3b-7c01-4f1a-8d6c-24bf24b2e5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:02.592155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bdd3b-7c01-4f1a-8d6c-24bf24b2e5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:02.592492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bdd3b-7c01-4f1a-8d6c-24bf24b2e5d2 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:02.593331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bdd3b-7c01-4f1a-8d6c-24bf24b2e5d2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:02.593680 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bdd3b-7c01-4f1a-8d6c-24bf24b2e5d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:02.595974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bdd3b-7c01-4f1a-8d6c-24bf24b2e5d2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:02.631479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 345/690] 10.222.0.42 () {56 vars in 1135 bytes} [Wed May 25 16:07:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:02.638358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bdd3b-7c01-4f1a-8d6c-24bf24b2e5d2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:02.702295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb736fa-8976-41e7-afcd-d99a077784b5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:02.733627 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:02.770122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238085db-afbf-4edf-acf6-cc92a2fd45ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:02.770837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238085db-afbf-4edf-acf6-cc92a2fd45ae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:02.771118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238085db-afbf-4edf-acf6-cc92a2fd45ae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:02.771392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238085db-afbf-4edf-acf6-cc92a2fd45ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:02.779340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:02.791413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238085db-afbf-4edf-acf6-cc92a2fd45ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:02.792893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238085db-afbf-4edf-acf6-cc92a2fd45ae None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-238085db-afbf-4edf-acf6-cc92a2fd45ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:02.830552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:02.831653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:02.832587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:02.832587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:02.832587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:02.832587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:02.832587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:02.832587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:02.832587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:02.832587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:02.832587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:02.832587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:02.833122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bdd3b-7c01-4f1a-8d6c-24bf24b2e5d2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:02.833206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 346/691] 10.222.0.42 () {56 vars in 1093 bytes} [Wed May 25 16:07:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:02.839265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238085db-afbf-4edf-acf6-cc92a2fd45ae None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:02.866764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c8253-845d-4085-96d1-c41b9f74f85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:02.867480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8253-845d-4085-96d1-c41b9f74f85b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:02.867759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8253-845d-4085-96d1-c41b9f74f85b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:02.868029 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8253-845d-4085-96d1-c41b9f74f85b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:02.944429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c8253-845d-4085-96d1-c41b9f74f85b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:02.949739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c8253-845d-4085-96d1-c41b9f74f85b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:03.034334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:03.061127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 346/692] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:07:03.066735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c8253-845d-4085-96d1-c41b9f74f85b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:03.136055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238085db-afbf-4edf-acf6-cc92a2fd45ae None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:03.182233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c8253-845d-4085-96d1-c41b9f74f85b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:03.223885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf849c-a83e-493a-983b-82d167371fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:03.224956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf849c-a83e-493a-983b-82d167371fdb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:03.225243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf849c-a83e-493a-983b-82d167371fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:03.225517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf849c-a83e-493a-983b-82d167371fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:03.237941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf849c-a83e-493a-983b-82d167371fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:03.239388 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf849c-a83e-493a-983b-82d167371fdb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:03.250459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c762b4-dbdd-4dd0-ae72-e542e09ffb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:03.251544 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c762b4-dbdd-4dd0-ae72-e542e09ffb21 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:03.252043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c762b4-dbdd-4dd0-ae72-e542e09ffb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:03.252533 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c762b4-dbdd-4dd0-ae72-e542e09ffb21 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:03.253585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c762b4-dbdd-4dd0-ae72-e542e09ffb21 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:03.254118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c762b4-dbdd-4dd0-ae72-e542e09ffb21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:03.255863 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c762b4-dbdd-4dd0-ae72-e542e09ffb21 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-82bf849c-a83e-493a-983b-82d167371fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:03.261765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:03.263153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:03.264049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:03.264049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:03.264049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:03.264049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:03.264049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 347/693] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:03.266657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf849c-a83e-493a-983b-82d167371fdb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:03.303017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 347/694] 10.222.0.42 () {56 vars in 1133 bytes} [Wed May 25 16:07:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:03.305769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c762b4-dbdd-4dd0-ae72-e542e09ffb21 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:03.314617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:03.384205 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:03.439054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf849c-a83e-493a-983b-82d167371fdb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:03.508463 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c762b4-dbdd-4dd0-ae72-e542e09ffb21 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:03.548633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57678d-d8d4-47db-8e4a-cff7d490a084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:03.549404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57678d-d8d4-47db-8e4a-cff7d490a084 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:03.549760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57678d-d8d4-47db-8e4a-cff7d490a084 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:03.549936 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57678d-d8d4-47db-8e4a-cff7d490a084 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:03.550969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57678d-d8d4-47db-8e4a-cff7d490a084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:03.551288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57678d-d8d4-47db-8e4a-cff7d490a084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:03.552918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57678d-d8d4-47db-8e4a-cff7d490a084 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:03.581491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 348/695] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:03.582968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff21fb-1ea0-410a-9f29-d5476b198d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:03.583622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff21fb-1ea0-410a-9f29-d5476b198d07 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:03.583899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff21fb-1ea0-410a-9f29-d5476b198d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:03.584170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff21fb-1ea0-410a-9f29-d5476b198d07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:03.586227 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57678d-d8d4-47db-8e4a-cff7d490a084 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:03.684160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:03.709888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57678d-d8d4-47db-8e4a-cff7d490a084 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:03.727414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff21fb-1ea0-410a-9f29-d5476b198d07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:03.728882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff21fb-1ea0-410a-9f29-d5476b198d07 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:03.752189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c24308-6c7e-49e1-998b-1697096c041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:03.752991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c24308-6c7e-49e1-998b-1697096c041e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:03.753269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c24308-6c7e-49e1-998b-1697096c041e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:03.753540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c24308-6c7e-49e1-998b-1697096c041e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:03.763361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c24308-6c7e-49e1-998b-1697096c041e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:03.765536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c24308-6c7e-49e1-998b-1697096c041e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:03.770761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d2c24308-6c7e-49e1-998b-1697096c041e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:03.789064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:03.790199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:03.791270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:03.791270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:03.791270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:03.791270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:03.791270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:03.791270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:03.791270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:03.791270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:03.794431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 349/696] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:03.795852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f1a2b-3fb5-4804-b5dc-f7ad887a918b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:03.796264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f1a2b-3fb5-4804-b5dc-f7ad887a918b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:03.796662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f1a2b-3fb5-4804-b5dc-f7ad887a918b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:03.798333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 350/697] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:03.803693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f1a2b-3fb5-4804-b5dc-f7ad887a918b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:03.864536 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 348/698] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:07:03.866199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500184f-2c3c-417d-89e0-69ed6ed2408c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:03.866630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500184f-2c3c-417d-89e0-69ed6ed2408c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:03.867050 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500184f-2c3c-417d-89e0-69ed6ed2408c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:03.888992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1500184f-2c3c-417d-89e0-69ed6ed2408c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:03.890488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1500184f-2c3c-417d-89e0-69ed6ed2408c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:03.928286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f1a2b-3fb5-4804-b5dc-f7ad887a918b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:03.963041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 349/699] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:07:03.964113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500184f-2c3c-417d-89e0-69ed6ed2408c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:03.972140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a64b7-b019-43a7-9a1f-941f0f9c1ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:03.972926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a64b7-b019-43a7-9a1f-941f0f9c1ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:03.973314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a64b7-b019-43a7-9a1f-941f0f9c1ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:03.973698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a64b7-b019-43a7-9a1f-941f0f9c1ab5 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:03.974620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a64b7-b019-43a7-9a1f-941f0f9c1ab5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:03.975036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a64b7-b019-43a7-9a1f-941f0f9c1ab5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:03.976869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a64b7-b019-43a7-9a1f-941f0f9c1ab5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:03.999618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:04.006930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 351/700] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:04.012661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a64b7-b019-43a7-9a1f-941f0f9c1ab5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:04.035642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:04.078271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500184f-2c3c-417d-89e0-69ed6ed2408c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:04.115440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2ed9e-bdc3-480f-b11a-4643cbfc4e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:04.116984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2ed9e-bdc3-480f-b11a-4643cbfc4e59 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.117349 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2ed9e-bdc3-480f-b11a-4643cbfc4e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.117604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2ed9e-bdc3-480f-b11a-4643cbfc4e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.126455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a64b7-b019-43a7-9a1f-941f0f9c1ab5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:04.129400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2ed9e-bdc3-480f-b11a-4643cbfc4e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:04.130920 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2ed9e-bdc3-480f-b11a-4643cbfc4e59 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-fac2ed9e-bdc3-480f-b11a-4643cbfc4e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:04.150090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:04.151088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:04.152042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:04.152042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:04.152042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:04.152042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:04.152042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 350/701] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:04.152406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2ed9e-bdc3-480f-b11a-4643cbfc4e59 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:04.172223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea10e7-803d-49d0-a7bf-328094b0dbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:04.172893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea10e7-803d-49d0-a7bf-328094b0dbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.173178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea10e7-803d-49d0-a7bf-328094b0dbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.173454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea10e7-803d-49d0-a7bf-328094b0dbc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.174477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea10e7-803d-49d0-a7bf-328094b0dbc4 None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=c6eb13611d0d4b6abe035c9bfbfde79c) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:04.174904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea10e7-803d-49d0-a7bf-328094b0dbc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:04.176322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea10e7-803d-49d0-a7bf-328094b0dbc4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:04.184839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 352/702] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:04 2022] GET /identity/v3/role_assignments?user.id=c6eb13611d0d4b6abe035c9bfbfde79c&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:04.186097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895a32f-d8d6-4213-83aa-63621f28ecd5 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.186382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895a32f-d8d6-4213-83aa-63621f28ecd5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.186662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895a32f-d8d6-4213-83aa-63621f28ecd5 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.192598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 353/703] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:04.193686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895a32f-d8d6-4213-83aa-63621f28ecd5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:04.275855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:04.289337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2ed9e-bdc3-480f-b11a-4643cbfc4e59 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:04.314806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:04.328120 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895a32f-d8d6-4213-83aa-63621f28ecd5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:04.340941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137e174-839e-494a-ae8f-e207fa149ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:04.342878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137e174-839e-494a-ae8f-e207fa149ace None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.343402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137e174-839e-494a-ae8f-e207fa149ace None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.343927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137e174-839e-494a-ae8f-e207fa149ace None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.345019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137e174-839e-494a-ae8f-e207fa149ace None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:04.347188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137e174-839e-494a-ae8f-e207fa149ace None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4137e174-839e-494a-ae8f-e207fa149ace None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:04.368813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:04.370075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 16:07:04.371104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:04.373562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 351/704] 10.222.0.42 () {56 vars in 1129 bytes} [Wed May 25 16:07:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:04.374349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197d487-787a-4721-b0f3-d26a070198a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:04.375062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197d487-787a-4721-b0f3-d26a070198a5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.375344 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197d487-787a-4721-b0f3-d26a070198a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.375616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197d487-787a-4721-b0f3-d26a070198a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.376342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197d487-787a-4721-b0f3-d26a070198a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:04.376648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197d487-787a-4721-b0f3-d26a070198a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:04.378386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb350124-3027-4071-8488-3267502f4bc8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.378669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb350124-3027-4071-8488-3267502f4bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.379402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb350124-3027-4071-8488-3267502f4bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.380151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197d487-787a-4721-b0f3-d26a070198a5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:04.396328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 354/705] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:04.397527 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197d487-787a-4721-b0f3-d26a070198a5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:04.398381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cb350124-3027-4071-8488-3267502f4bc8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:04.399811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cb350124-3027-4071-8488-3267502f4bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:04.489508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 352/706] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:07:04.490592 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb350124-3027-4071-8488-3267502f4bc8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:04.532977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197d487-787a-4721-b0f3-d26a070198a5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:04.560754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:04.574995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84affb10-942a-48ba-a417-368feed8c6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:04.575843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84affb10-942a-48ba-a417-368feed8c6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.576783 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84affb10-942a-48ba-a417-368feed8c6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.577165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84affb10-942a-48ba-a417-368feed8c6e2 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.577956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84affb10-942a-48ba-a417-368feed8c6e2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:04.578318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84affb10-942a-48ba-a417-368feed8c6e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:04.580240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84affb10-942a-48ba-a417-368feed8c6e2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:04.587430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 355/707] 10.222.0.42 () {56 vars in 1124 bytes} [Wed May 25 16:07:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:04.588544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84affb10-942a-48ba-a417-368feed8c6e2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:04.612273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb350124-3027-4071-8488-3267502f4bc8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:04.627840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:04.661668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119b3b6-ff83-4c98-ba1a-6b47afc740bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:04.662497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119b3b6-ff83-4c98-ba1a-6b47afc740bd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.662841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119b3b6-ff83-4c98-ba1a-6b47afc740bd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.663123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119b3b6-ff83-4c98-ba1a-6b47afc740bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.673467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119b3b6-ff83-4c98-ba1a-6b47afc740bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:04.675907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119b3b6-ff83-4c98-ba1a-6b47afc740bd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9119b3b6-ff83-4c98-ba1a-6b47afc740bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:04.694552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:04.695512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:04.696407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:04.696407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:04.696407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:04.696407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:04.696407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:04.696407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:04.696407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:04.696407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:04.697160 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 353/708] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:04.701743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119b3b6-ff83-4c98-ba1a-6b47afc740bd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:04.725115 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84affb10-942a-48ba-a417-368feed8c6e2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:04.765651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b971ea-3144-4168-9bb6-0f4b2c3fc19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:04.766686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b971ea-3144-4168-9bb6-0f4b2c3fc19a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.767185 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b971ea-3144-4168-9bb6-0f4b2c3fc19a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.767528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b971ea-3144-4168-9bb6-0f4b2c3fc19a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.779867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b971ea-3144-4168-9bb6-0f4b2c3fc19a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:04.781608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b971ea-3144-4168-9bb6-0f4b2c3fc19a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-77b971ea-3144-4168-9bb6-0f4b2c3fc19a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:04.804982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:04.806856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:04.809038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:04.809038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:04.809038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:04.809038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:04.809038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 356/709] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:04.809038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b971ea-3144-4168-9bb6-0f4b2c3fc19a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:04.833460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119b3b6-ff83-4c98-ba1a-6b47afc740bd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:04.867524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:04.870993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea292fe-fba3-4e19-a14b-64e8481b66c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:04.871816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea292fe-fba3-4e19-a14b-64e8481b66c5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.872183 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea292fe-fba3-4e19-a14b-64e8481b66c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.872351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea292fe-fba3-4e19-a14b-64e8481b66c5 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.873226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea292fe-fba3-4e19-a14b-64e8481b66c5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:04.873543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea292fe-fba3-4e19-a14b-64e8481b66c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:04.875116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea292fe-fba3-4e19-a14b-64e8481b66c5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:04.881752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 354/710] 10.222.0.42 () {56 vars in 1124 bytes} [Wed May 25 16:07:04 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:04.882610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea292fe-fba3-4e19-a14b-64e8481b66c5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:04.934606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b971ea-3144-4168-9bb6-0f4b2c3fc19a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:04.951738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:04.975079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a24974-efb8-427c-890e-b9b9c4571ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:04.975751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a24974-efb8-427c-890e-b9b9c4571ecc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.976037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a24974-efb8-427c-890e-b9b9c4571ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.976312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a24974-efb8-427c-890e-b9b9c4571ecc None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.977113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a24974-efb8-427c-890e-b9b9c4571ecc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:04.977422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a24974-efb8-427c-890e-b9b9c4571ecc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:04.978929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a24974-efb8-427c-890e-b9b9c4571ecc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:04.993792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 357/711] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:04.994133 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9f503-f5c2-4085-9325-ac1c5db19de9 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:04.994421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9f503-f5c2-4085-9325-ac1c5db19de9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:04.994698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9f503-f5c2-4085-9325-ac1c5db19de9 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:04.996363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 358/712] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:04.998179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9f503-f5c2-4085-9325-ac1c5db19de9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:05.044145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea292fe-fba3-4e19-a14b-64e8481b66c5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:05.093708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da58ee2-bb32-4252-a89f-0eae64083bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:05.094457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58ee2-bb32-4252-a89f-0eae64083bdc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:05.095102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58ee2-bb32-4252-a89f-0eae64083bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:05.095398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58ee2-bb32-4252-a89f-0eae64083bdc None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:05.096223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da58ee2-bb32-4252-a89f-0eae64083bdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:05.096824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da58ee2-bb32-4252-a89f-0eae64083bdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:05.098309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da58ee2-bb32-4252-a89f-0eae64083bdc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:05.118028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 355/713] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:05.118219 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:05.119317 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229a08d-1c35-435e-804c-f112bd7cc43c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:05.119604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229a08d-1c35-435e-804c-f112bd7cc43c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:05.119877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229a08d-1c35-435e-804c-f112bd7cc43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:05.126174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9f503-f5c2-4085-9325-ac1c5db19de9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:05.141065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c229a08d-1c35-435e-804c-f112bd7cc43c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:05.142267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c229a08d-1c35-435e-804c-f112bd7cc43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:05.173099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0cce8-38ee-490f-987e-0108e635d331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:05.174060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0cce8-38ee-490f-987e-0108e635d331 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:05.174397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0cce8-38ee-490f-987e-0108e635d331 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:05.174675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0cce8-38ee-490f-987e-0108e635d331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:05.176218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0cce8-38ee-490f-987e-0108e635d331 None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=6e6795b5b5cf4499ad494b875756ba71) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:05.176656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0cce8-38ee-490f-987e-0108e635d331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:05.178387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0cce8-38ee-490f-987e-0108e635d331 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:05.186571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 359/714] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:04 2022] GET /identity/v3/role_assignments?user.id=6e6795b5b5cf4499ad494b875756ba71&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:05.187583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0cce8-38ee-490f-987e-0108e635d331 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:05.218357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 356/715] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:07:05.254849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229a08d-1c35-435e-804c-f112bd7cc43c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:05.289188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:05.328750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0cce8-38ee-490f-987e-0108e635d331 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:05.371671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a3af8-e24d-42ae-895f-53d09601e191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:05.372534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a3af8-e24d-42ae-895f-53d09601e191 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:05.372915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a3af8-e24d-42ae-895f-53d09601e191 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:05.373314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a3af8-e24d-42ae-895f-53d09601e191 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:05.374299 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a3af8-e24d-42ae-895f-53d09601e191 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:05.376054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a3af8-e24d-42ae-895f-53d09601e191 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:05.382240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229a08d-1c35-435e-804c-f112bd7cc43c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:05.410025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 360/716] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:07:05 2022] POST /identity/v3/users => generated 254 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:05.444096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62c678-707e-4aa3-9787-07a7bb8dc8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:05.444933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62c678-707e-4aa3-9787-07a7bb8dc8be None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:05.445218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62c678-707e-4aa3-9787-07a7bb8dc8be None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:05.445496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62c678-707e-4aa3-9787-07a7bb8dc8be None admin] PATH_INFO: `/v3/services/network` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:05.446201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62c678-707e-4aa3-9787-07a7bb8dc8be None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:05.447654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62c678-707e-4aa3-9787-07a7bb8dc8be None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-fe62c678-707e-4aa3-9787-07a7bb8dc8be None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:05.474734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:05.475782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:05.477206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 357/717] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:05 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:05.482674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a3af8-e24d-42ae-895f-53d09601e191 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:05.522096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:05.634119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a3af8-e24d-42ae-895f-53d09601e191 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:05.673000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c50e3-c2e3-420b-b262-f150e45731ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:05.674060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c50e3-c2e3-420b-b262-f150e45731ca None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:05.674562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c50e3-c2e3-420b-b262-f150e45731ca None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:05.675042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c50e3-c2e3-420b-b262-f150e45731ca None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:05.675950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c50e3-c2e3-420b-b262-f150e45731ca None admin] RBAC: Adding query filter params (name=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:05.676455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c50e3-c2e3-420b-b262-f150e45731ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:05.678037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c50e3-c2e3-420b-b262-f150e45731ca None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:05.683380 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 361/718] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:05.696368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62c678-707e-4aa3-9787-07a7bb8dc8be None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:05.816006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:05.830149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62c678-707e-4aa3-9787-07a7bb8dc8be None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:05.869412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc17bc-1444-49df-b4f7-c01e70f1d501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:05.870483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc17bc-1444-49df-b4f7-c01e70f1d501 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:05.871049 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc17bc-1444-49df-b4f7-c01e70f1d501 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:05.871557 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc17bc-1444-49df-b4f7-c01e70f1d501 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:05.872548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc17bc-1444-49df-b4f7-c01e70f1d501 None admin] RBAC: Adding query filter params (type=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:05.873054 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc17bc-1444-49df-b4f7-c01e70f1d501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:05.874694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc17bc-1444-49df-b4f7-c01e70f1d501 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:05.882203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 358/719] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:06.821446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545d0d-828d-409c-a088-100c47147d98 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:06.822001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545d0d-828d-409c-a088-100c47147d98 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:06.822001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545d0d-828d-409c-a088-100c47147d98 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:06.823609 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 362/720] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:06.829394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeb7b6-d9d6-4bad-a143-32c9324ce889 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:06.830261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeb7b6-d9d6-4bad-a143-32c9324ce889 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:06.830744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeb7b6-d9d6-4bad-a143-32c9324ce889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:06.848794 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f8eeb7b6-d9d6-4bad-a143-32c9324ce889 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:06.850398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f8eeb7b6-d9d6-4bad-a143-32c9324ce889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:06.949797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 359/721] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:07:06.983948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f545d0d-828d-409c-a088-100c47147d98 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:07.101422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:07.119655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f545d0d-828d-409c-a088-100c47147d98 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:07.179876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee1c52-23eb-45a1-b0a6-d97a23cdfefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:07.180795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee1c52-23eb-45a1-b0a6-d97a23cdfefc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:07.181094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee1c52-23eb-45a1-b0a6-d97a23cdfefc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:07.181421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee1c52-23eb-45a1-b0a6-d97a23cdfefc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:07.202675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee1c52-23eb-45a1-b0a6-d97a23cdfefc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-59ee1c52-23eb-45a1-b0a6-d97a23cdfefc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:07.210347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:07.211297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:07.212144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 363/722] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:07.217804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeb7b6-d9d6-4bad-a143-32c9324ce889 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:07.326607 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:07.354344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeb7b6-d9d6-4bad-a143-32c9324ce889 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:07.396809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa7371-19cb-4bae-82f7-2ace013eb27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:07.397507 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa7371-19cb-4bae-82f7-2ace013eb27f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:07.397812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa7371-19cb-4bae-82f7-2ace013eb27f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:07.398090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa7371-19cb-4bae-82f7-2ace013eb27f None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:07.398860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa7371-19cb-4bae-82f7-2ace013eb27f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:07.399166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa7371-19cb-4bae-82f7-2ace013eb27f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:07.400550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa7371-19cb-4bae-82f7-2ace013eb27f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:07.414001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 360/723] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:07:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:07.420415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee1c52-23eb-45a1-b0a6-d97a23cdfefc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:07.629668 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee1c52-23eb-45a1-b0a6-d97a23cdfefc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:07.683851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a8bea-1416-4f33-a017-981539b3a51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:07.684933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a8bea-1416-4f33-a017-981539b3a51c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:07.685213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a8bea-1416-4f33-a017-981539b3a51c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:07.685487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a8bea-1416-4f33-a017-981539b3a51c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:07.707166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a8bea-1416-4f33-a017-981539b3a51c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:07.708692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a8bea-1416-4f33-a017-981539b3a51c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:07.711958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-bc1a8bea-1416-4f33-a017-981539b3a51c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:07.733079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:07.734148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:07.735013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:07.735013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:07.735013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:07.735013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:07.735013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 364/724] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:07.741286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa7371-19cb-4bae-82f7-2ace013eb27f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:07.884229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa7371-19cb-4bae-82f7-2ace013eb27f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:07.928888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9a948-08ce-433c-bd97-b6de26d55876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:07.929883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9a948-08ce-433c-bd97-b6de26d55876 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:07.930315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9a948-08ce-433c-bd97-b6de26d55876 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:07.930738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9a948-08ce-433c-bd97-b6de26d55876 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:07.931619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9a948-08ce-433c-bd97-b6de26d55876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:07.932074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9a948-08ce-433c-bd97-b6de26d55876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:07.933820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9a948-08ce-433c-bd97-b6de26d55876 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:07.953506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 361/725] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:07.958754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a8bea-1416-4f33-a017-981539b3a51c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:08.004393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bb570-f3c7-48af-9566-f621beaeecf5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.004947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bb570-f3c7-48af-9566-f621beaeecf5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.005390 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bb570-f3c7-48af-9566-f621beaeecf5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:08.007212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 362/726] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:08.017585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab86d9f-b37b-415a-b890-5f8f01660b48 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.017910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab86d9f-b37b-415a-b890-5f8f01660b48 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.018187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab86d9f-b37b-415a-b890-5f8f01660b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:08.028161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:08.044169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fab86d9f-b37b-415a-b890-5f8f01660b48 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:08.045758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fab86d9f-b37b-415a-b890-5f8f01660b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:08.111920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a8bea-1416-4f33-a017-981539b3a51c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:08.128913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 363/727] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:07:08.130634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b80c51-78bd-45d4-b5a0-f6a5b7d68a09 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.131137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b80c51-78bd-45d4-b5a0-f6a5b7d68a09 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.131594 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b80c51-78bd-45d4-b5a0-f6a5b7d68a09 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:08.133759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 364/728] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:08.139816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f174af5-e021-4689-9183-be79af45ce28 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.140332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f174af5-e021-4689-9183-be79af45ce28 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.140807 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f174af5-e021-4689-9183-be79af45ce28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:08.179086 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10e079-c22d-4afb-ba91-8b8213b4c151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:08.179926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10e079-c22d-4afb-ba91-8b8213b4c151 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.180212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10e079-c22d-4afb-ba91-8b8213b4c151 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.180492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10e079-c22d-4afb-ba91-8b8213b4c151 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:08.191100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10e079-c22d-4afb-ba91-8b8213b4c151 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:08.192855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10e079-c22d-4afb-ba91-8b8213b4c151 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3e10e079-c22d-4afb-ba91-8b8213b4c151 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:08.209376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:08.210667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:08.211876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:08.211876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:08.211876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:08.211876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:08.211876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:08.211876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:08.211876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:08.211876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:08.211876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 16:07:08.211876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:08.214521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 365/729] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:08.215767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10e079-c22d-4afb-ba91-8b8213b4c151 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:08.282504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:08.290675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1f174af5-e021-4689-9183-be79af45ce28 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:08.291865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1f174af5-e021-4689-9183-be79af45ce28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:08.338024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10e079-c22d-4afb-ba91-8b8213b4c151 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:08.366926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 365/730] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 16:07:08.368093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f174af5-e021-4689-9183-be79af45ce28 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:08.395113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529201a-5ed1-4c32-9b75-7da79f68bccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:08.395980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529201a-5ed1-4c32-9b75-7da79f68bccd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.396448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529201a-5ed1-4c32-9b75-7da79f68bccd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.396899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529201a-5ed1-4c32-9b75-7da79f68bccd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:08.410970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529201a-5ed1-4c32-9b75-7da79f68bccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b529201a-5ed1-4c32-9b75-7da79f68bccd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:08.419686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:08.420908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:08.422021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:08.424440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 366/731] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:08.426047 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4510183-adbf-491a-b67a-ad590a9cad40 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.426335 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4510183-adbf-491a-b67a-ad590a9cad40 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.426615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4510183-adbf-491a-b67a-ad590a9cad40 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:08.432270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 367/732] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:08.432270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4510183-adbf-491a-b67a-ad590a9cad40 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:08.454983 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:08.501218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f174af5-e021-4689-9183-be79af45ce28 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:08.545546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f718e-ef92-4eac-a0b2-b56db6385c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:08.546249 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f718e-ef92-4eac-a0b2-b56db6385c9f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.546537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f718e-ef92-4eac-a0b2-b56db6385c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.546817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f718e-ef92-4eac-a0b2-b56db6385c9f None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:08.547670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f718e-ef92-4eac-a0b2-b56db6385c9f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:08.547979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f718e-ef92-4eac-a0b2-b56db6385c9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:08.549437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f718e-ef92-4eac-a0b2-b56db6385c9f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:08.562711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4510183-adbf-491a-b67a-ad590a9cad40 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:08.573569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 366/733] 10.222.0.42 () {56 vars in 1135 bytes} [Wed May 25 16:07:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:08.578811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f718e-ef92-4eac-a0b2-b56db6385c9f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:08.587810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:08.609815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e3fe8-f064-42c9-a882-d64156c11716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:08.610494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e3fe8-f064-42c9-a882-d64156c11716 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.610775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e3fe8-f064-42c9-a882-d64156c11716 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.611056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e3fe8-f064-42c9-a882-d64156c11716 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:08.611776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e3fe8-f064-42c9-a882-d64156c11716 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:08.613180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e3fe8-f064-42c9-a882-d64156c11716 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:08.629699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 368/734] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:07:08 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:08.635709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6ae30-02b1-4b8c-b4cd-45d073e46b91 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.636065 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6ae30-02b1-4b8c-b4cd-45d073e46b91 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.636399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6ae30-02b1-4b8c-b4cd-45d073e46b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:08.663449 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b8f6ae30-02b1-4b8c-b4cd-45d073e46b91 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:08.664969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b8f6ae30-02b1-4b8c-b4cd-45d073e46b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:08.708420 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f718e-ef92-4eac-a0b2-b56db6385c9f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:08.743809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:08.768361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e2272-7e94-4b5f-84b9-ffa819ed31da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:08.769625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e2272-7e94-4b5f-84b9-ffa819ed31da None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.777940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 369/735] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:07:08.777940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6ae30-02b1-4b8c-b4cd-45d073e46b91 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:08.779125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e2272-7e94-4b5f-84b9-ffa819ed31da None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.779683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e2272-7e94-4b5f-84b9-ffa819ed31da None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:08.780804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e2272-7e94-4b5f-84b9-ffa819ed31da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:08.781380 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e2272-7e94-4b5f-84b9-ffa819ed31da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:08.783157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e2272-7e94-4b5f-84b9-ffa819ed31da None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:08.803312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 367/736] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:07:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:08.808988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e2272-7e94-4b5f-84b9-ffa819ed31da None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:08.929442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6ae30-02b1-4b8c-b4cd-45d073e46b91 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:08.941520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:08.965768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e2272-7e94-4b5f-84b9-ffa819ed31da None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:08.988272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73457a3-1e32-43a8-b71d-b719f3784ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:08.989571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73457a3-1e32-43a8-b71d-b719f3784ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:08.990025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73457a3-1e32-43a8-b71d-b719f3784ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:08.990445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73457a3-1e32-43a8-b71d-b719f3784ed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:09.014135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92a8b7-c1c4-4d78-a0df-3ee7e7b52485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:09.014940 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92a8b7-c1c4-4d78-a0df-3ee7e7b52485 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:09.015232 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92a8b7-c1c4-4d78-a0df-3ee7e7b52485 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:09.015989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92a8b7-c1c4-4d78-a0df-3ee7e7b52485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:09.027663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92a8b7-c1c4-4d78-a0df-3ee7e7b52485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:09.029364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92a8b7-c1c4-4d78-a0df-3ee7e7b52485 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-af92a8b7-c1c4-4d78-a0df-3ee7e7b52485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:09.048530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:09.050175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:09.051158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:09.051158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:09.051158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:09.051158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:09.054137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 368/737] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:09.055844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92a8b7-c1c4-4d78-a0df-3ee7e7b52485 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:09.085151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73457a3-1e32-43a8-b71d-b719f3784ed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:09.087090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73457a3-1e32-43a8-b71d-b719f3784ed0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:09.093192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:09.169529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92a8b7-c1c4-4d78-a0df-3ee7e7b52485 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:09.208664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144803c-cf50-4d48-987f-fbcb2320b711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:09.209385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144803c-cf50-4d48-987f-fbcb2320b711 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:09.209668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144803c-cf50-4d48-987f-fbcb2320b711 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:09.209978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144803c-cf50-4d48-987f-fbcb2320b711 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:09.211099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 370/738] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:08 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 16:07:09.212871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7ccc7-223f-4914-8b86-20f0108c6408 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:09.213229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7ccc7-223f-4914-8b86-20f0108c6408 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:09.213569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7ccc7-223f-4914-8b86-20f0108c6408 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:09.215206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 371/739] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:09.216330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7ccc7-223f-4914-8b86-20f0108c6408 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:09.227997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144803c-cf50-4d48-987f-fbcb2320b711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:09.229209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f144803c-cf50-4d48-987f-fbcb2320b711 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:09.236394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:09.237375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:09.238260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 369/740] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:07:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:09.239261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144803c-cf50-4d48-987f-fbcb2320b711 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:09.342301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:09.345111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7ccc7-223f-4914-8b86-20f0108c6408 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:09.352052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144803c-cf50-4d48-987f-fbcb2320b711 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:09.384933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ca43f-b4cc-4478-9eba-9175f3463f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:09.385654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ca43f-b4cc-4478-9eba-9175f3463f2f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:09.385966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ca43f-b4cc-4478-9eba-9175f3463f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:09.386239 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ca43f-b4cc-4478-9eba-9175f3463f2f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:09.387961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ca43f-b4cc-4478-9eba-9175f3463f2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:09.388338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ca43f-b4cc-4478-9eba-9175f3463f2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:09.389760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ca43f-b4cc-4478-9eba-9175f3463f2f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:09.393279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5fb78-3e04-43b8-8779-8fc028d6d656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:09.394317 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5fb78-3e04-43b8-8779-8fc028d6d656 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:09.394807 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5fb78-3e04-43b8-8779-8fc028d6d656 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:09.395263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5fb78-3e04-43b8-8779-8fc028d6d656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:09.403083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 372/741] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:09.404716 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e42dcd-7a48-4e18-9e1e-16840ff361c0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:09.405000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e42dcd-7a48-4e18-9e1e-16840ff361c0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:09.405272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e42dcd-7a48-4e18-9e1e-16840ff361c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:09.414035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5fb78-3e04-43b8-8779-8fc028d6d656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:09.415610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5fb78-3e04-43b8-8779-8fc028d6d656 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:09.422985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c3e42dcd-7a48-4e18-9e1e-16840ff361c0 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:09.424176 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c3e42dcd-7a48-4e18-9e1e-16840ff361c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a8f5fb78-3e04-43b8-8779-8fc028d6d656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:09.440166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:09.441085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:09.442025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:09.442025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:09.442025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:09.442025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:09.442025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 370/742] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:09.442305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5fb78-3e04-43b8-8779-8fc028d6d656 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:09.493747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:09.502513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 373/743] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 16:07:09.503530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e42dcd-7a48-4e18-9e1e-16840ff361c0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:09.575494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5fb78-3e04-43b8-8779-8fc028d6d656 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:09.615819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c6d6b-87a6-4ad9-93f4-25beafa2848e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:09.616489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c6d6b-87a6-4ad9-93f4-25beafa2848e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:09.616775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c6d6b-87a6-4ad9-93f4-25beafa2848e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:09.617042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c6d6b-87a6-4ad9-93f4-25beafa2848e None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:09.617833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c6d6b-87a6-4ad9-93f4-25beafa2848e None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:09.618137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c6d6b-87a6-4ad9-93f4-25beafa2848e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:09.619534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c6d6b-87a6-4ad9-93f4-25beafa2848e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:09.628083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e42dcd-7a48-4e18-9e1e-16840ff361c0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:09.629443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 371/744] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:07:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:09.634677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c6d6b-87a6-4ad9-93f4-25beafa2848e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:09.645507 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:09.670774 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a770c7-08bc-4286-b3f7-5ffe940e0026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:09.671578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a770c7-08bc-4286-b3f7-5ffe940e0026 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:09.671864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a770c7-08bc-4286-b3f7-5ffe940e0026 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:09.672149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a770c7-08bc-4286-b3f7-5ffe940e0026 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:09.679562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a770c7-08bc-4286-b3f7-5ffe940e0026 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:09.680996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a770c7-08bc-4286-b3f7-5ffe940e0026 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-32a770c7-08bc-4286-b3f7-5ffe940e0026 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:09.696593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:09.697559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 374/745] 10.222.0.42 () {56 vars in 1093 bytes} [Wed May 25 16:07:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:09.698535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a770c7-08bc-4286-b3f7-5ffe940e0026 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:09.754509 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c6d6b-87a6-4ad9-93f4-25beafa2848e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:09.762533 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:09.798007 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658b883-a0a9-4f94-93b1-92c3d4d297b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:09.799102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658b883-a0a9-4f94-93b1-92c3d4d297b5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:09.799281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658b883-a0a9-4f94-93b1-92c3d4d297b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:09.799584 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658b883-a0a9-4f94-93b1-92c3d4d297b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:09.800284 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658b883-a0a9-4f94-93b1-92c3d4d297b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:09.800591 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658b883-a0a9-4f94-93b1-92c3d4d297b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:09.801991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658b883-a0a9-4f94-93b1-92c3d4d297b5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:09.813562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a770c7-08bc-4286-b3f7-5ffe940e0026 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:09.814427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 372/746] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:09.815300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658b883-a0a9-4f94-93b1-92c3d4d297b5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:09.857364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db644ef5-9909-4ef7-b2ca-d53e142b55bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:09.857753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db644ef5-9909-4ef7-b2ca-d53e142b55bb None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:09.858059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db644ef5-9909-4ef7-b2ca-d53e142b55bb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:09.858334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db644ef5-9909-4ef7-b2ca-d53e142b55bb None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:09.859058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db644ef5-9909-4ef7-b2ca-d53e142b55bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:09.860457 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db644ef5-9909-4ef7-b2ca-d53e142b55bb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:09.885753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 375/747] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:07:09 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:09.885753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db644ef5-9909-4ef7-b2ca-d53e142b55bb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:09.907781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:09.956812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658b883-a0a9-4f94-93b1-92c3d4d297b5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:10.004388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c6bac-7b4b-45ae-8e24-5daa8f429c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:10.005350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c6bac-7b4b-45ae-8e24-5daa8f429c1b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:10.005638 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c6bac-7b4b-45ae-8e24-5daa8f429c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:10.006031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c6bac-7b4b-45ae-8e24-5daa8f429c1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:10.019015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c6bac-7b4b-45ae-8e24-5daa8f429c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:10.020655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c6bac-7b4b-45ae-8e24-5daa8f429c1b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:10.024892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db644ef5-9909-4ef7-b2ca-d53e142b55bb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a19c6bac-7b4b-45ae-8e24-5daa8f429c1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:10.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:10.043459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:10.044456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:10.044456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:10.044456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:10.044456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:10.044686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 373/748] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:10.045064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:10.046996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c6bac-7b4b-45ae-8e24-5daa8f429c1b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:10.069356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6dd7-4eb9-4977-a69e-7b9ec780e0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:10.070316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6dd7-4eb9-4977-a69e-7b9ec780e0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:10.070785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6dd7-4eb9-4977-a69e-7b9ec780e0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:10.071166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6dd7-4eb9-4977-a69e-7b9ec780e0a1 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:10.072125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe6dd7-4eb9-4977-a69e-7b9ec780e0a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:10.072543 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe6dd7-4eb9-4977-a69e-7b9ec780e0a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:10.074555 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe6dd7-4eb9-4977-a69e-7b9ec780e0a1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:10.094712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 376/749] 10.222.0.42 () {56 vars in 1133 bytes} [Wed May 25 16:07:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:10.095705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6dd7-4eb9-4977-a69e-7b9ec780e0a1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:10.173041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c6bac-7b4b-45ae-8e24-5daa8f429c1b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:10.193445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:10.216767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21425073-80f5-45ad-98f0-bb0866bac950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:10.217577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21425073-80f5-45ad-98f0-bb0866bac950 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:10.217892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21425073-80f5-45ad-98f0-bb0866bac950 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:10.218132 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21425073-80f5-45ad-98f0-bb0866bac950 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:10.225477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21425073-80f5-45ad-98f0-bb0866bac950 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:10.227052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21425073-80f5-45ad-98f0-bb0866bac950 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:10.232034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6dd7-4eb9-4977-a69e-7b9ec780e0a1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-21425073-80f5-45ad-98f0-bb0866bac950 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:10.245379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:10.246948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:10.247856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:10.247856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:10.247856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:10.247856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:10.247856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:10.247856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:10.247856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:10.247856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:10.250952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 374/750] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:10.252121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21425073-80f5-45ad-98f0-bb0866bac950 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:10.276650 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cd1e7-8b7d-41f4-8198-ec88c6e24fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:10.277296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cd1e7-8b7d-41f4-8198-ec88c6e24fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:10.278138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cd1e7-8b7d-41f4-8198-ec88c6e24fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:10.278422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cd1e7-8b7d-41f4-8198-ec88c6e24fa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:10.279660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cd1e7-8b7d-41f4-8198-ec88c6e24fa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:10.279968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cd1e7-8b7d-41f4-8198-ec88c6e24fa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:10.281889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cd1e7-8b7d-41f4-8198-ec88c6e24fa7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:10.299319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 377/751] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:10.300324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cd1e7-8b7d-41f4-8198-ec88c6e24fa7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:10.340007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:10.380100 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21425073-80f5-45ad-98f0-bb0866bac950 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:10.416774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba4c4c-5b93-419d-9169-5a1284ede9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:10.418178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba4c4c-5b93-419d-9169-5a1284ede9ac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:10.418561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba4c4c-5b93-419d-9169-5a1284ede9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:10.418679 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cd1e7-8b7d-41f4-8198-ec88c6e24fa7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:10.418931 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba4c4c-5b93-419d-9169-5a1284ede9ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:10.431548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba4c4c-5b93-419d-9169-5a1284ede9ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:10.432098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba4c4c-5b93-419d-9169-5a1284ede9ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1aba4c4c-5b93-419d-9169-5a1284ede9ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:10.451578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:10.452451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:10.453397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:10.453397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:10.453397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:10.453397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:10.456939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c12bc7-8ad0-4bb1-854d-8893a1598d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:10.457108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 375/752] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:10.457836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c12bc7-8ad0-4bb1-854d-8893a1598d21 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:10.457953 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba4c4c-5b93-419d-9169-5a1284ede9ac None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:10.458408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c12bc7-8ad0-4bb1-854d-8893a1598d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:10.458779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c12bc7-8ad0-4bb1-854d-8893a1598d21 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:10.459719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c12bc7-8ad0-4bb1-854d-8893a1598d21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:10.460138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c12bc7-8ad0-4bb1-854d-8893a1598d21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:10.461869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c12bc7-8ad0-4bb1-854d-8893a1598d21 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:10.476644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 378/753] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:10.477625 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c12bc7-8ad0-4bb1-854d-8893a1598d21 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:10.494229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:10.575326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba4c4c-5b93-419d-9169-5a1284ede9ac None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:10.591175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c12bc7-8ad0-4bb1-854d-8893a1598d21 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:10.609751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:10.613007 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd50e75-8adf-4f34-a656-c3601ab688f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:10.613859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd50e75-8adf-4f34-a656-c3601ab688f7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:10.614177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd50e75-8adf-4f34-a656-c3601ab688f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:10.614472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd50e75-8adf-4f34-a656-c3601ab688f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:10.625027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd50e75-8adf-4f34-a656-c3601ab688f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:10.626618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd50e75-8adf-4f34-a656-c3601ab688f7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:10.631055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30911728-e1f3-4356-84c2-f25dff8ce0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:10.631931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30911728-e1f3-4356-84c2-f25dff8ce0eb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:10.632368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30911728-e1f3-4356-84c2-f25dff8ce0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:10.632793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30911728-e1f3-4356-84c2-f25dff8ce0eb None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:10.633660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30911728-e1f3-4356-84c2-f25dff8ce0eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:10.634146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30911728-e1f3-4356-84c2-f25dff8ce0eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:10.635662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30911728-e1f3-4356-84c2-f25dff8ce0eb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2bd50e75-8adf-4f34-a656-c3601ab688f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:10.642251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:10.643216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:10.644182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:10.644182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:10.644182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:10.644182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:10.644182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:10.644182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:10.644182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:10.644182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:10.644182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:10.644182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:10.644182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 376/754] 10.222.0.42 () {56 vars in 1117 bytes} [Wed May 25 16:07:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:10.645303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd50e75-8adf-4f34-a656-c3601ab688f7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:10.651797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 379/755] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:10.653514 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30911728-e1f3-4356-84c2-f25dff8ce0eb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:10.747500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:10.765927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd50e75-8adf-4f34-a656-c3601ab688f7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:10.769571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30911728-e1f3-4356-84c2-f25dff8ce0eb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:10.806819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d4464-b01b-4a45-ad0d-5daccdbdf158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:10.807595 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d4464-b01b-4a45-ad0d-5daccdbdf158 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:10.807948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d4464-b01b-4a45-ad0d-5daccdbdf158 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:10.808896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d4464-b01b-4a45-ad0d-5daccdbdf158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:10.810378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d4464-b01b-4a45-ad0d-5daccdbdf158 None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=c6eb13611d0d4b6abe035c9bfbfde79c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:10.810768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d4464-b01b-4a45-ad0d-5daccdbdf158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:10.812012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2e5ce-1758-40c5-ba6e-f4814f0c874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:10.812420 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d4464-b01b-4a45-ad0d-5daccdbdf158 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:10.813119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2e5ce-1758-40c5-ba6e-f4814f0c874c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:10.813585 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2e5ce-1758-40c5-ba6e-f4814f0c874c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:10.814082 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2e5ce-1758-40c5-ba6e-f4814f0c874c None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:10.815282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2e5ce-1758-40c5-ba6e-f4814f0c874c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:10.815801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2e5ce-1758-40c5-ba6e-f4814f0c874c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:10.817567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2e5ce-1758-40c5-ba6e-f4814f0c874c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:10.825991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 377/756] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:10 2022] GET /identity/v3/role_assignments?user.id=c6eb13611d0d4b6abe035c9bfbfde79c&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:10.825991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d4464-b01b-4a45-ad0d-5daccdbdf158 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:10.853397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 380/757] 10.222.0.42 () {56 vars in 1149 bytes} [Wed May 25 16:07:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:10.858955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2e5ce-1758-40c5-ba6e-f4814f0c874c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:10.884231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:10.944969 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d4464-b01b-4a45-ad0d-5daccdbdf158 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:11.010037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2e5ce-1758-40c5-ba6e-f4814f0c874c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:11.010234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08c469-ef32-4ebd-8003-7cca00213967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:11.010969 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08c469-ef32-4ebd-8003-7cca00213967 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:11.012028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08c469-ef32-4ebd-8003-7cca00213967 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:11.012582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08c469-ef32-4ebd-8003-7cca00213967 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:11.020629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08c469-ef32-4ebd-8003-7cca00213967 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:11.025904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08c469-ef32-4ebd-8003-7cca00213967 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1b08c469-ef32-4ebd-8003-7cca00213967 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:11.042063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:11.043567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:11.044470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:11.044470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:11.044470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:11.044470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:11.044470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:11.044470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:11.044470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:11.044470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:11.046373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 378/758] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:11.048162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08c469-ef32-4ebd-8003-7cca00213967 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:11.062049 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:11.076844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab60c0e-389e-4673-9087-5191fb2e151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:11.077426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab60c0e-389e-4673-9087-5191fb2e151e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:11.077735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab60c0e-389e-4673-9087-5191fb2e151e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:11.078017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab60c0e-389e-4673-9087-5191fb2e151e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:11.161344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08c469-ef32-4ebd-8003-7cca00213967 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:11.169973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab60c0e-389e-4673-9087-5191fb2e151e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:11.171459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab60c0e-389e-4673-9087-5191fb2e151e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:11.193621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:11.205673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd1255-8d1e-44d7-8dba-f9f1514aad90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:11.206549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd1255-8d1e-44d7-8dba-f9f1514aad90 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:11.206914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd1255-8d1e-44d7-8dba-f9f1514aad90 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:11.207334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd1255-8d1e-44d7-8dba-f9f1514aad90 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:11.208236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd1255-8d1e-44d7-8dba-f9f1514aad90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:11.208649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd1255-8d1e-44d7-8dba-f9f1514aad90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:11.210241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd1255-8d1e-44d7-8dba-f9f1514aad90 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:11.227041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 379/759] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:11.227041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c797311-73f2-4d95-b2e5-2da50bec97a7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:11.227041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c797311-73f2-4d95-b2e5-2da50bec97a7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:11.227041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c797311-73f2-4d95-b2e5-2da50bec97a7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:11.231996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 380/760] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:11.233413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124077c-8649-4e9c-9b56-85a04ca45ed3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:11.234047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124077c-8649-4e9c-9b56-85a04ca45ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:11.234438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124077c-8649-4e9c-9b56-85a04ca45ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:11.252388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e124077c-8649-4e9c-9b56-85a04ca45ed3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:11.253781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e124077c-8649-4e9c-9b56-85a04ca45ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:11.321385 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 381/761] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 16:07:11.322538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab60c0e-389e-4673-9087-5191fb2e151e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:11.332030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:11.341834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 381/762] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:11.343075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124077c-8649-4e9c-9b56-85a04ca45ed3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:11.477576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124077c-8649-4e9c-9b56-85a04ca45ed3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:11.478371 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab60c0e-389e-4673-9087-5191fb2e151e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:11.502705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:11.516645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fd48d-a9b2-4606-9b7f-6acba03e8a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:11.517628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fd48d-a9b2-4606-9b7f-6acba03e8a81 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:11.518039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fd48d-a9b2-4606-9b7f-6acba03e8a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:11.518461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fd48d-a9b2-4606-9b7f-6acba03e8a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:11.530167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fd48d-a9b2-4606-9b7f-6acba03e8a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:11.531784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fd48d-a9b2-4606-9b7f-6acba03e8a81 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:11.534392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e05107-5c8b-4ecd-bc45-052834d0d726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:11.535907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e05107-5c8b-4ecd-bc45-052834d0d726 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:11.536369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e05107-5c8b-4ecd-bc45-052834d0d726 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:11.536790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e05107-5c8b-4ecd-bc45-052834d0d726 None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/6e6795b5b5cf4499ad494b875756ba71/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:11.538969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e05107-5c8b-4ecd-bc45-052834d0d726 None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=6e6795b5b5cf4499ad494b875756ba71, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:11.541102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e05107-5c8b-4ecd-bc45-052834d0d726 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-717fd48d-a9b2-4606-9b7f-6acba03e8a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:11.552510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:11.553651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:11.554791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:11.554791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:11.554791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:11.554791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:11.556361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 382/763] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:11.557394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fd48d-a9b2-4606-9b7f-6acba03e8a81 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:11.565779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 382/764] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:07:11 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/6e6795b5b5cf4499ad494b875756ba71/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:11.565779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e05107-5c8b-4ecd-bc45-052834d0d726 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:11.678748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:11.732659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e05107-5c8b-4ecd-bc45-052834d0d726 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:11.788315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a4bfd-e054-458f-a58f-63028a5d0981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:11.789001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a4bfd-e054-458f-a58f-63028a5d0981 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:11.789291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a4bfd-e054-458f-a58f-63028a5d0981 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:11.791141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a4bfd-e054-458f-a58f-63028a5d0981 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:11.792144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a4bfd-e054-458f-a58f-63028a5d0981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:11.792459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a4bfd-e054-458f-a58f-63028a5d0981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:11.794535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a4bfd-e054-458f-a58f-63028a5d0981 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:11.800912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fd48d-a9b2-4606-9b7f-6acba03e8a81 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:11.831927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 383/765] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:11.839092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a4bfd-e054-458f-a58f-63028a5d0981 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:11.851677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffee60e-955f-4c81-a8f3-22ba4989c5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:11.852493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffee60e-955f-4c81-a8f3-22ba4989c5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:11.852777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffee60e-955f-4c81-a8f3-22ba4989c5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:11.853055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffee60e-955f-4c81-a8f3-22ba4989c5f2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:11.853762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffee60e-955f-4c81-a8f3-22ba4989c5f2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:11.855765 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffee60e-955f-4c81-a8f3-22ba4989c5f2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7ffee60e-955f-4c81-a8f3-22ba4989c5f2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:11.869457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:11.870491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:11.872080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 383/766] 10.222.0.42 () {56 vars in 1129 bytes} [Wed May 25 16:07:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:11.878006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffee60e-955f-4c81-a8f3-22ba4989c5f2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:11.884630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:11.983024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a4bfd-e054-458f-a58f-63028a5d0981 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:12.000495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffee60e-955f-4c81-a8f3-22ba4989c5f2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:12.021229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:12.033520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68018294-be55-4d71-bbd1-27993e2204d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:12.034620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68018294-be55-4d71-bbd1-27993e2204d5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:12.034903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68018294-be55-4d71-bbd1-27993e2204d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:12.035226 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68018294-be55-4d71-bbd1-27993e2204d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:12.042393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68018294-be55-4d71-bbd1-27993e2204d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:12.043908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68018294-be55-4d71-bbd1-27993e2204d5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:12.047742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81104f74-5e59-49ad-a318-d53fb37c15f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:12.048733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81104f74-5e59-49ad-a318-d53fb37c15f4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:12.049211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81104f74-5e59-49ad-a318-d53fb37c15f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:12.049689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81104f74-5e59-49ad-a318-d53fb37c15f4 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:12.050905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81104f74-5e59-49ad-a318-d53fb37c15f4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:12.051411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81104f74-5e59-49ad-a318-d53fb37c15f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:12.052974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81104f74-5e59-49ad-a318-d53fb37c15f4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-68018294-be55-4d71-bbd1-27993e2204d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:12.059179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:12.060070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:12.060957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:12.060957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:12.060957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:12.060957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:12.060957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:12.060957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:12.060957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:12.060957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:12.061396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 384/767] 10.222.0.42 () {56 vars in 1124 bytes} [Wed May 25 16:07:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:12.065773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 384/768] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:12.067792 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81104f74-5e59-49ad-a318-d53fb37c15f4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:12.073361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68018294-be55-4d71-bbd1-27993e2204d5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:12.290489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81104f74-5e59-49ad-a318-d53fb37c15f4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:12.322000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68018294-be55-4d71-bbd1-27993e2204d5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:12.326077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:12.369762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31157c-8ea5-4e59-9ff4-dfd98f73e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:12.369762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31157c-8ea5-4e59-9ff4-dfd98f73e2ef None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:12.369762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31157c-8ea5-4e59-9ff4-dfd98f73e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:12.369762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31157c-8ea5-4e59-9ff4-dfd98f73e2ef None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:12.370245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31157c-8ea5-4e59-9ff4-dfd98f73e2ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:12.370245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31157c-8ea5-4e59-9ff4-dfd98f73e2ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:12.370245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31157c-8ea5-4e59-9ff4-dfd98f73e2ef None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:12.378532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3155440-1d46-47a0-9adf-a095594c0b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:12.379936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3155440-1d46-47a0-9adf-a095594c0b47 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:12.380640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3155440-1d46-47a0-9adf-a095594c0b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:12.381067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3155440-1d46-47a0-9adf-a095594c0b47 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:12.381950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3155440-1d46-47a0-9adf-a095594c0b47 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:12.382403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3155440-1d46-47a0-9adf-a095594c0b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:12.382875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 385/769] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:12.384126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3155440-1d46-47a0-9adf-a095594c0b47 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:12.392729 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31157c-8ea5-4e59-9ff4-dfd98f73e2ef None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:12.394238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 385/770] 10.222.0.42 () {56 vars in 1124 bytes} [Wed May 25 16:07:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:12.401177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3155440-1d46-47a0-9adf-a095594c0b47 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:12.453816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:12.540597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3155440-1d46-47a0-9adf-a095594c0b47 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:12.547970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31157c-8ea5-4e59-9ff4-dfd98f73e2ef None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:12.583777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d858b-2b5a-4919-afda-6192c50d9f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:12.584640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d858b-2b5a-4919-afda-6192c50d9f34 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:12.585019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d858b-2b5a-4919-afda-6192c50d9f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:12.585305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d858b-2b5a-4919-afda-6192c50d9f34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:12.586113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eda7e-aa20-4939-8110-76c9674dd431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:12.586249 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d858b-2b5a-4919-afda-6192c50d9f34 None admin] RBAC: Adding query filter params (interface=public, service_id=a64df37793674b608d245055142a36ce, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:12.586535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d858b-2b5a-4919-afda-6192c50d9f34 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:12.586948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eda7e-aa20-4939-8110-76c9674dd431 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:12.587296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eda7e-aa20-4939-8110-76c9674dd431 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:12.587645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eda7e-aa20-4939-8110-76c9674dd431 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:12.587907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d858b-2b5a-4919-afda-6192c50d9f34 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:12.588611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eda7e-aa20-4939-8110-76c9674dd431 None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=730cb89f7a214fde9c5040674323f8db) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:12.588991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eda7e-aa20-4939-8110-76c9674dd431 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:12.590658 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eda7e-aa20-4939-8110-76c9674dd431 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:12.596881 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 386/771] 10.222.0.42 () {56 vars in 1251 bytes} [Wed May 25 16:07:12 2022] GET /identity/v3/endpoints?service_id=a64df37793674b608d245055142a36ce&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:12.598491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388763b3-31f8-4a0c-82fa-48cc441f71f7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:12.598793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388763b3-31f8-4a0c-82fa-48cc441f71f7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:12.599077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388763b3-31f8-4a0c-82fa-48cc441f71f7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:12.601515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 386/773] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:12 2022] GET /identity/v3/role_assignments?user.id=730cb89f7a214fde9c5040674323f8db&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:12.602597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eda7e-aa20-4939-8110-76c9674dd431 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:12.604749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 387/773] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:12.606765 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5310c-ea51-4afd-a366-fc0cbf35f69d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:12.607062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5310c-ea51-4afd-a366-fc0cbf35f69d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:12.607352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5310c-ea51-4afd-a366-fc0cbf35f69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:12.625675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b6b5310c-ea51-4afd-a366-fc0cbf35f69d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:12.626768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:12.627753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b6b5310c-ea51-4afd-a366-fc0cbf35f69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:12.711163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 388/774] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:12.725174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eda7e-aa20-4939-8110-76c9674dd431 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:12.735370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5310c-ea51-4afd-a366-fc0cbf35f69d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:12.764093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46b81b-fbe9-41c9-ac22-77e410a72fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:12.764836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46b81b-fbe9-41c9-ac22-77e410a72fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:12.765124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46b81b-fbe9-41c9-ac22-77e410a72fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:12.765402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46b81b-fbe9-41c9-ac22-77e410a72fe9 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:12.766165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46b81b-fbe9-41c9-ac22-77e410a72fe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:12.767631 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46b81b-fbe9-41c9-ac22-77e410a72fe9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:12.773833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 387/775] 10.222.0.42 () {56 vars in 1087 bytes} [Wed May 25 16:07:12 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:12.814886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:12.905005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5310c-ea51-4afd-a366-fc0cbf35f69d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:12.952698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab62e40-5e21-4193-8456-fcf96752f19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:12.953741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab62e40-5e21-4193-8456-fcf96752f19f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:12.954142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab62e40-5e21-4193-8456-fcf96752f19f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:12.954519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab62e40-5e21-4193-8456-fcf96752f19f None admin] PATH_INFO: `/v3/services/network` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:12.955310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab62e40-5e21-4193-8456-fcf96752f19f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:12.956856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab62e40-5e21-4193-8456-fcf96752f19f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2ab62e40-5e21-4193-8456-fcf96752f19f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:12.970279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:12.971571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 16:07:12.972636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:12.979450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 389/776] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:12 2022] GET /identity/v3/services/network => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:12.985417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46b81b-fbe9-41c9-ac22-77e410a72fe9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:13.104372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:13.126054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46b81b-fbe9-41c9-ac22-77e410a72fe9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:13.173055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffba9fb-93a0-4f78-8683-17797a7e6bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:13.173828 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffba9fb-93a0-4f78-8683-17797a7e6bee None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:13.174119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffba9fb-93a0-4f78-8683-17797a7e6bee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:13.174449 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffba9fb-93a0-4f78-8683-17797a7e6bee None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:13.175172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffba9fb-93a0-4f78-8683-17797a7e6bee None admin] RBAC: Adding query filter params (name=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:13.175483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffba9fb-93a0-4f78-8683-17797a7e6bee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:13.176867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffba9fb-93a0-4f78-8683-17797a7e6bee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:13.182427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 388/777] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:13.188548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab62e40-5e21-4193-8456-fcf96752f19f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:13.302775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab62e40-5e21-4193-8456-fcf96752f19f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:13.304346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:13.340477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c251eec-260f-411a-9d6c-27593ca4cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:13.341158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c251eec-260f-411a-9d6c-27593ca4cb97 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:13.341446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c251eec-260f-411a-9d6c-27593ca4cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:13.341743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c251eec-260f-411a-9d6c-27593ca4cb97 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:13.342468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c251eec-260f-411a-9d6c-27593ca4cb97 None admin] RBAC: Adding query filter params (type=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:13.342776 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c251eec-260f-411a-9d6c-27593ca4cb97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:13.344158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c251eec-260f-411a-9d6c-27593ca4cb97 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:13.350115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 390/778] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:13.356703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffba9fb-93a0-4f78-8683-17797a7e6bee None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:13.358922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a267a-0c7b-499e-8cf6-e3dffbeb5a57 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:13.359233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a267a-0c7b-499e-8cf6-e3dffbeb5a57 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:13.359613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a267a-0c7b-499e-8cf6-e3dffbeb5a57 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:13.361465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 391/779] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:13.374054 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fd765-44bb-4a45-a376-bcefb460d2ea None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:13.374440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fd765-44bb-4a45-a376-bcefb460d2ea None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:13.374720 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fd765-44bb-4a45-a376-bcefb460d2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:13.406604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a67fd765-44bb-4a45-a376-bcefb460d2ea None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:13.407811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a67fd765-44bb-4a45-a376-bcefb460d2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:13.538442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 392/780] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:13.556800 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffba9fb-93a0-4f78-8683-17797a7e6bee None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:13.574188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fd765-44bb-4a45-a376-bcefb460d2ea None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:13.637029 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d538e71-947a-4655-a6b6-00ec92455baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:13.637635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d538e71-947a-4655-a6b6-00ec92455baf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:13.637952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d538e71-947a-4655-a6b6-00ec92455baf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:13.638229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d538e71-947a-4655-a6b6-00ec92455baf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:13.639031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d538e71-947a-4655-a6b6-00ec92455baf None admin] RBAC: Adding query filter params (interface=public, service_id=a209bc8aacba465ea154e2a5a8c1e2f5, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:13.639346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d538e71-947a-4655-a6b6-00ec92455baf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:13.640743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d538e71-947a-4655-a6b6-00ec92455baf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:13.659678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 389/781] 10.222.0.42 () {56 vars in 1251 bytes} [Wed May 25 16:07:13 2022] GET /identity/v3/endpoints?service_id=a209bc8aacba465ea154e2a5a8c1e2f5&interface=public®ion_id=RegionOne => generated 194 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:13.665975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d538e71-947a-4655-a6b6-00ec92455baf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:13.681284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:13.744628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:13.754629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fd765-44bb-4a45-a376-bcefb460d2ea None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:13.828233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf9fed-b337-482b-9fd1-adb4fa8c8024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:13.829049 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf9fed-b337-482b-9fd1-adb4fa8c8024 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:13.829333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf9fed-b337-482b-9fd1-adb4fa8c8024 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:13.829611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf9fed-b337-482b-9fd1-adb4fa8c8024 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:13.866531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf9fed-b337-482b-9fd1-adb4fa8c8024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0fcf9fed-b337-482b-9fd1-adb4fa8c8024 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:13.884346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:13.885393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:13.886834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 393/782] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:13.893028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf9fed-b337-482b-9fd1-adb4fa8c8024 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:13.986329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d538e71-947a-4655-a6b6-00ec92455baf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:14.035488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93635179-4caa-489f-9c0a-eb3170c037a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:14.036807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93635179-4caa-489f-9c0a-eb3170c037a6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:14.037324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93635179-4caa-489f-9c0a-eb3170c037a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:14.037832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93635179-4caa-489f-9c0a-eb3170c037a6 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:14.038008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf9fed-b337-482b-9fd1-adb4fa8c8024 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:14.038889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93635179-4caa-489f-9c0a-eb3170c037a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:14.041691 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93635179-4caa-489f-9c0a-eb3170c037a6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:14.050178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 390/783] 10.222.0.42 () {56 vars in 1087 bytes} [Wed May 25 16:07:13 2022] GET /identity/v3/services => generated 772 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:14.081300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f670e-0e34-4381-88d3-75cb8898afc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:14.082449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f670e-0e34-4381-88d3-75cb8898afc0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:14.082945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f670e-0e34-4381-88d3-75cb8898afc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:14.083423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f670e-0e34-4381-88d3-75cb8898afc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:14.084415 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f670e-0e34-4381-88d3-75cb8898afc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:14.084929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f670e-0e34-4381-88d3-75cb8898afc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:14.086573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f670e-0e34-4381-88d3-75cb8898afc0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:14.096697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 394/784] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:07:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:14.102399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93635179-4caa-489f-9c0a-eb3170c037a6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:14.206392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:14.239072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93635179-4caa-489f-9c0a-eb3170c037a6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:14.288029 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b7088-7efe-44d2-b1c7-deb3cddc7b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:14.288848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b7088-7efe-44d2-b1c7-deb3cddc7b39 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:14.289133 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b7088-7efe-44d2-b1c7-deb3cddc7b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:14.289411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b7088-7efe-44d2-b1c7-deb3cddc7b39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:14.305551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b7088-7efe-44d2-b1c7-deb3cddc7b39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:14.307024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b7088-7efe-44d2-b1c7-deb3cddc7b39 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-258b7088-7efe-44d2-b1c7-deb3cddc7b39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:14.326654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:14.327603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:14.328489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:14.328489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:14.328489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:14.328489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:14.328489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 391/785] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:14.338821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f670e-0e34-4381-88d3-75cb8898afc0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:14.361578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:14.456859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f670e-0e34-4381-88d3-75cb8898afc0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:14.499301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cb6f5-2878-4ed3-8eea-9b99e20fb179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:14.500006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cb6f5-2878-4ed3-8eea-9b99e20fb179 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:14.500294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cb6f5-2878-4ed3-8eea-9b99e20fb179 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:14.500575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cb6f5-2878-4ed3-8eea-9b99e20fb179 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:14.501335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cb6f5-2878-4ed3-8eea-9b99e20fb179 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:14.501645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cb6f5-2878-4ed3-8eea-9b99e20fb179 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:14.503118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cb6f5-2878-4ed3-8eea-9b99e20fb179 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:14.521138 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 395/786] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:14.526544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b7088-7efe-44d2-b1c7-deb3cddc7b39 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:14.676118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b7088-7efe-44d2-b1c7-deb3cddc7b39 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:14.700939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:14.717898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f8704-e97b-4e0e-9c91-a1ef7a4cefeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:14.717898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8704-e97b-4e0e-9c91-a1ef7a4cefeb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:14.717898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8704-e97b-4e0e-9c91-a1ef7a4cefeb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:14.719263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8704-e97b-4e0e-9c91-a1ef7a4cefeb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:14.735158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f8704-e97b-4e0e-9c91-a1ef7a4cefeb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:14.736619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f8704-e97b-4e0e-9c91-a1ef7a4cefeb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-be9f8704-e97b-4e0e-9c91-a1ef7a4cefeb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:14.752169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:14.753149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:14.755510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:14.755510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:14.755510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:14.755510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:14.755510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:14.755510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:14.755510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:14.755510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:14.755510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 16:07:14.755510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:14.755510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 392/787] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:14.766870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cb6f5-2878-4ed3-8eea-9b99e20fb179 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:14.812472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad27cc8-a539-42a5-8fa4-4b0ac35e9745 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:14.812794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad27cc8-a539-42a5-8fa4-4b0ac35e9745 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:14.813035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad27cc8-a539-42a5-8fa4-4b0ac35e9745 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:14.814702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 393/788] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:07:14.822274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28d437-2eab-4cc0-8fa4-71c3a752d837 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:14.822564 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28d437-2eab-4cc0-8fa4-71c3a752d837 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:14.822842 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28d437-2eab-4cc0-8fa4-71c3a752d837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:14.848905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5c28d437-2eab-4cc0-8fa4-71c3a752d837 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:14.858484 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5c28d437-2eab-4cc0-8fa4-71c3a752d837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:14.867359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:14.903813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cb6f5-2878-4ed3-8eea-9b99e20fb179 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:14.958839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 394/789] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:14.970222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a16a7d-37a4-49a8-9581-a757ecd0bfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:14.970834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a16a7d-37a4-49a8-9581-a757ecd0bfbc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:14.971102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a16a7d-37a4-49a8-9581-a757ecd0bfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:14.971387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a16a7d-37a4-49a8-9581-a757ecd0bfbc None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:14.972238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a16a7d-37a4-49a8-9581-a757ecd0bfbc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:14.972548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a16a7d-37a4-49a8-9581-a757ecd0bfbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:14.974010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a16a7d-37a4-49a8-9581-a757ecd0bfbc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:15.006300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 396/790] 10.222.0.42 () {56 vars in 1135 bytes} [Wed May 25 16:07:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:15.007261 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28d437-2eab-4cc0-8fa4-71c3a752d837 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:15.009784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a16a7d-37a4-49a8-9581-a757ecd0bfbc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:15.121741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a16a7d-37a4-49a8-9581-a757ecd0bfbc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:15.157992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672f484-2636-4a0e-8fd7-9610fc7a0684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:15.159052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672f484-2636-4a0e-8fd7-9610fc7a0684 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:15.159377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672f484-2636-4a0e-8fd7-9610fc7a0684 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:15.159653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672f484-2636-4a0e-8fd7-9610fc7a0684 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:15.162496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:15.171717 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672f484-2636-4a0e-8fd7-9610fc7a0684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b672f484-2636-4a0e-8fd7-9610fc7a0684 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:15.180305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:15.181278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:15.182155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:15.185515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 397/791] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:15.190443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672f484-2636-4a0e-8fd7-9610fc7a0684 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:15.224194 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28d437-2eab-4cc0-8fa4-71c3a752d837 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:15.244412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:15.270151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a20eb-ab10-4ac7-9070-ca87faed52b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:15.271049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a20eb-ab10-4ac7-9070-ca87faed52b5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:15.271413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a20eb-ab10-4ac7-9070-ca87faed52b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:15.271763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a20eb-ab10-4ac7-9070-ca87faed52b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:15.284400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a20eb-ab10-4ac7-9070-ca87faed52b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:15.286002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a20eb-ab10-4ac7-9070-ca87faed52b5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-258a20eb-ab10-4ac7-9070-ca87faed52b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:15.308833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:15.309928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:15.310822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:15.310822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:15.310822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:15.310822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:15.310822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 395/792] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:15.316411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2eea-9491-4c97-9af1-b6a2f01565fe None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:15.316769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2eea-9491-4c97-9af1-b6a2f01565fe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:15.317125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2eea-9491-4c97-9af1-b6a2f01565fe None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:15.318102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672f484-2636-4a0e-8fd7-9610fc7a0684 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:15.318901 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 396/793] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:15.319712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d2eea-9491-4c97-9af1-b6a2f01565fe None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:15.362033 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3a916-2408-44cb-bffe-c7fb2fca2586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:15.362829 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3a916-2408-44cb-bffe-c7fb2fca2586 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:15.363185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3a916-2408-44cb-bffe-c7fb2fca2586 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:15.363573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3a916-2408-44cb-bffe-c7fb2fca2586 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:15.364623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3a916-2408-44cb-bffe-c7fb2fca2586 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:15.364939 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3a916-2408-44cb-bffe-c7fb2fca2586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:15.366449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3a916-2408-44cb-bffe-c7fb2fca2586 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:15.380933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 398/794] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:07:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:15.382279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930350d7-78a3-41d9-bf89-7301861772b3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:15.382576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930350d7-78a3-41d9-bf89-7301861772b3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:15.382860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930350d7-78a3-41d9-bf89-7301861772b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:15.403387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-930350d7-78a3-41d9-bf89-7301861772b3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:15.404600 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-930350d7-78a3-41d9-bf89-7301861772b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:15.465401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:15.468093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d2eea-9491-4c97-9af1-b6a2f01565fe None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:15.486494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 399/795] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:15.486494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930350d7-78a3-41d9-bf89-7301861772b3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:15.531269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fa19d-b085-4de6-a232-4c44ed4a2acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:15.532557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fa19d-b085-4de6-a232-4c44ed4a2acd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:15.533147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fa19d-b085-4de6-a232-4c44ed4a2acd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:15.534024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fa19d-b085-4de6-a232-4c44ed4a2acd None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:15.534935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365fa19d-b085-4de6-a232-4c44ed4a2acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:15.535343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365fa19d-b085-4de6-a232-4c44ed4a2acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:15.536814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365fa19d-b085-4de6-a232-4c44ed4a2acd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:15.550149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 397/796] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:15.555550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fa19d-b085-4de6-a232-4c44ed4a2acd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:15.586627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:15.634174 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930350d7-78a3-41d9-bf89-7301861772b3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:15.672652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fa19d-b085-4de6-a232-4c44ed4a2acd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:15.683734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477de44-3f72-4af4-9f2b-1516fe35d3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:15.685317 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477de44-3f72-4af4-9f2b-1516fe35d3fd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:15.685923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477de44-3f72-4af4-9f2b-1516fe35d3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:15.686444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477de44-3f72-4af4-9f2b-1516fe35d3fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:15.701008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477de44-3f72-4af4-9f2b-1516fe35d3fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:15.703258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477de44-3f72-4af4-9f2b-1516fe35d3fd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:15.711927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9efa9-83e3-4e3b-8e16-fd96689c7f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:15.712931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9efa9-83e3-4e3b-8e16-fd96689c7f1d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:15.713327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9efa9-83e3-4e3b-8e16-fd96689c7f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:15.713708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9efa9-83e3-4e3b-8e16-fd96689c7f1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:15.725060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9efa9-83e3-4e3b-8e16-fd96689c7f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6477de44-3f72-4af4-9f2b-1516fe35d3fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:15.725827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:15.726861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:15.727801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:15.727801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:15.727801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:15.727801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:15.729837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 400/797] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:15.731777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477de44-3f72-4af4-9f2b-1516fe35d3fd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-afb9efa9-83e3-4e3b-8e16-fd96689c7f1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:15.738586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:15.739476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:07:15.740328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:15.748830 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 398/798] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:07:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:15.750566 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9efa9-83e3-4e3b-8e16-fd96689c7f1d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:15.787032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:15.853272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477de44-3f72-4af4-9f2b-1516fe35d3fd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:15.861613 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:15.877600 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9efa9-83e3-4e3b-8e16-fd96689c7f1d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:15.890604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46c1f9-02b2-4636-be3d-c53edf7a19f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:15.891396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46c1f9-02b2-4636-be3d-c53edf7a19f0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:15.891690 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46c1f9-02b2-4636-be3d-c53edf7a19f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:15.892192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46c1f9-02b2-4636-be3d-c53edf7a19f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:15.899030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46c1f9-02b2-4636-be3d-c53edf7a19f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:15.900458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46c1f9-02b2-4636-be3d-c53edf7a19f0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2f46c1f9-02b2-4636-be3d-c53edf7a19f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:15.915442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:15.916430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:15.917354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:15.917354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:15.917354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:15.917354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:15.917354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:15.917354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:15.917354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:15.917354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:15.918308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880c3f1-325d-463e-8554-af74a148cd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:15.919095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880c3f1-325d-463e-8554-af74a148cd37 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:15.919378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880c3f1-325d-463e-8554-af74a148cd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:15.919650 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880c3f1-325d-463e-8554-af74a148cd37 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:15.920379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880c3f1-325d-463e-8554-af74a148cd37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:15.920683 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880c3f1-325d-463e-8554-af74a148cd37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:15.921846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 401/799] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:15.922760 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46c1f9-02b2-4636-be3d-c53edf7a19f0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:15.923786 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880c3f1-325d-463e-8554-af74a148cd37 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:15.937666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 399/800] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:15.939016 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89615407-2611-450b-8d21-e901a915b1ad None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:15.939309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89615407-2611-450b-8d21-e901a915b1ad None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:15.939576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89615407-2611-450b-8d21-e901a915b1ad None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:15.945087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 400/801] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:15.945087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89615407-2611-450b-8d21-e901a915b1ad None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:16.040085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:16.043301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46c1f9-02b2-4636-be3d-c53edf7a19f0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:16.055685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89615407-2611-450b-8d21-e901a915b1ad None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:16.082883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512deaa-bef3-4aaf-90ea-0240f7869407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:16.083917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512deaa-bef3-4aaf-90ea-0240f7869407 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.084453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512deaa-bef3-4aaf-90ea-0240f7869407 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.084908 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512deaa-bef3-4aaf-90ea-0240f7869407 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:16.085920 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512deaa-bef3-4aaf-90ea-0240f7869407 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:16.086376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512deaa-bef3-4aaf-90ea-0240f7869407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:16.088069 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512deaa-bef3-4aaf-90ea-0240f7869407 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:16.092520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f5bde-7248-4693-9673-47fc959b4374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:16.093195 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f5bde-7248-4693-9673-47fc959b4374 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.093483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f5bde-7248-4693-9673-47fc959b4374 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.093787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f5bde-7248-4693-9673-47fc959b4374 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:16.094609 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f5bde-7248-4693-9673-47fc959b4374 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:16.094921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f5bde-7248-4693-9673-47fc959b4374 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:16.096347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f5bde-7248-4693-9673-47fc959b4374 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:16.100457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 402/802] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:07:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:16.101511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512deaa-bef3-4aaf-90ea-0240f7869407 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:16.115614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 401/803] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:16.117319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b763-6a42-49d4-b217-d121686c8a9d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.117747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b763-6a42-49d4-b217-d121686c8a9d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.117937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b763-6a42-49d4-b217-d121686c8a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:16.130009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:16.136832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bce6b763-6a42-49d4-b217-d121686c8a9d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:16.138194 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bce6b763-6a42-49d4-b217-d121686c8a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:16.208712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 402/804] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:16.209670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6b763-6a42-49d4-b217-d121686c8a9d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:16.231568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512deaa-bef3-4aaf-90ea-0240f7869407 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:16.274961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62617b-ea82-4a66-8d10-be5a523b269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:16.276012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62617b-ea82-4a66-8d10-be5a523b269f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.276012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62617b-ea82-4a66-8d10-be5a523b269f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.276306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62617b-ea82-4a66-8d10-be5a523b269f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:16.286866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62617b-ea82-4a66-8d10-be5a523b269f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:16.288297 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62617b-ea82-4a66-8d10-be5a523b269f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9a62617b-ea82-4a66-8d10-be5a523b269f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:16.303630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:16.304866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:16.305837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:16.305837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:16.305837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:16.305837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:16.305837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:16.305837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:16.305837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:16.305837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:16.305837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:16.305837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:16.305837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 403/805] 10.222.0.42 () {56 vars in 1093 bytes} [Wed May 25 16:07:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:16.306438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62617b-ea82-4a66-8d10-be5a523b269f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:16.338266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:16.338763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6b763-6a42-49d4-b217-d121686c8a9d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:16.377674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f3940-a148-4ebf-aa67-30f1231e5a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:16.378497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f3940-a148-4ebf-aa67-30f1231e5a22 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.378775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f3940-a148-4ebf-aa67-30f1231e5a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.379050 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f3940-a148-4ebf-aa67-30f1231e5a22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:16.382618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:16.392812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f3940-a148-4ebf-aa67-30f1231e5a22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:16.394293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f3940-a148-4ebf-aa67-30f1231e5a22 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1c3f3940-a148-4ebf-aa67-30f1231e5a22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:16.413860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:16.414799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:16.415692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:16.415692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:16.415692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:16.415692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:16.420098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 403/806] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:16.420834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f3940-a148-4ebf-aa67-30f1231e5a22 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:16.435912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62617b-ea82-4a66-8d10-be5a523b269f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:16.475822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d132e16-0862-4d5c-b742-b354bdd9ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:16.477104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d132e16-0862-4d5c-b742-b354bdd9ca2f None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.477458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d132e16-0862-4d5c-b742-b354bdd9ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.477761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d132e16-0862-4d5c-b742-b354bdd9ca2f None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/c6eb13611d0d4b6abe035c9bfbfde79c/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:16.479731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d132e16-0862-4d5c-b742-b354bdd9ca2f None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=c6eb13611d0d4b6abe035c9bfbfde79c, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:16.481213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d132e16-0862-4d5c-b742-b354bdd9ca2f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:16.494853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 404/807] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:07:16 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/c6eb13611d0d4b6abe035c9bfbfde79c/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:16.496228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d132e16-0862-4d5c-b742-b354bdd9ca2f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:16.543103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f3940-a148-4ebf-aa67-30f1231e5a22 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:16.593278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47922e3f-f986-44be-a900-a7cf0a0c75fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:16.594542 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47922e3f-f986-44be-a900-a7cf0a0c75fa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.595023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47922e3f-f986-44be-a900-a7cf0a0c75fa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.595490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47922e3f-f986-44be-a900-a7cf0a0c75fa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:16.596377 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47922e3f-f986-44be-a900-a7cf0a0c75fa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:16.598725 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47922e3f-f986-44be-a900-a7cf0a0c75fa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:16.605643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-47922e3f-f986-44be-a900-a7cf0a0c75fa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:16.618073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:16.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 16:07:16.619941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:16.625855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 404/808] 10.222.0.42 () {56 vars in 1129 bytes} [Wed May 25 16:07:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:16.626962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47922e3f-f986-44be-a900-a7cf0a0c75fa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:16.639451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d132e16-0862-4d5c-b742-b354bdd9ca2f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:16.680714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d8bc3-1dd4-4a60-ba6e-ca20a4a3d018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:16.681842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d8bc3-1dd4-4a60-ba6e-ca20a4a3d018 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.682337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d8bc3-1dd4-4a60-ba6e-ca20a4a3d018 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.682816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d8bc3-1dd4-4a60-ba6e-ca20a4a3d018 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:16.685011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d8bc3-1dd4-4a60-ba6e-ca20a4a3d018 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:16.685664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d8bc3-1dd4-4a60-ba6e-ca20a4a3d018 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:16.687494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d8bc3-1dd4-4a60-ba6e-ca20a4a3d018 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:16.700200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:16.716959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 405/809] 10.222.0.42 () {56 vars in 1133 bytes} [Wed May 25 16:07:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:16.718631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d8bc3-1dd4-4a60-ba6e-ca20a4a3d018 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:16.763402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47922e3f-f986-44be-a900-a7cf0a0c75fa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:16.802149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870f1c1-407f-4efa-8d4e-f1cb5897699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:16.802822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870f1c1-407f-4efa-8d4e-f1cb5897699f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.803103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870f1c1-407f-4efa-8d4e-f1cb5897699f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.803381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870f1c1-407f-4efa-8d4e-f1cb5897699f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:16.804100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870f1c1-407f-4efa-8d4e-f1cb5897699f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:16.804403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870f1c1-407f-4efa-8d4e-f1cb5897699f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:16.805787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870f1c1-407f-4efa-8d4e-f1cb5897699f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:16.818835 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 405/810] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:16.819670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870f1c1-407f-4efa-8d4e-f1cb5897699f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:16.841345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d8bc3-1dd4-4a60-ba6e-ca20a4a3d018 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:16.877968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefba104-b124-4ff4-a4ae-67830f465c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:16.878555 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefba104-b124-4ff4-a4ae-67830f465c2b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.878839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefba104-b124-4ff4-a4ae-67830f465c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.879115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefba104-b124-4ff4-a4ae-67830f465c2b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:16.879830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefba104-b124-4ff4-a4ae-67830f465c2b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:16.880139 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefba104-b124-4ff4-a4ae-67830f465c2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:16.881515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefba104-b124-4ff4-a4ae-67830f465c2b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:16.887327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 406/811] 10.222.0.42 () {56 vars in 1124 bytes} [Wed May 25 16:07:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:16.888645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5100a9-7797-4621-ab76-edd16133fb25 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.888936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5100a9-7797-4621-ab76-edd16133fb25 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.889214 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5100a9-7797-4621-ab76-edd16133fb25 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:16.890656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 407/812] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:16.895408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5100a9-7797-4621-ab76-edd16133fb25 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:16.900118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:16.938991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870f1c1-407f-4efa-8d4e-f1cb5897699f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:16.953472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:16.976569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa31680-e963-48c1-8cb2-41a3f9c61838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:16.977344 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa31680-e963-48c1-8cb2-41a3f9c61838 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:16.977638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa31680-e963-48c1-8cb2-41a3f9c61838 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:16.977964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa31680-e963-48c1-8cb2-41a3f9c61838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:17.010028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5100a9-7797-4621-ab76-edd16133fb25 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:17.051354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb10fba-ea36-43eb-bda4-a8af12345235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:17.052167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb10fba-ea36-43eb-bda4-a8af12345235 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:17.052449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb10fba-ea36-43eb-bda4-a8af12345235 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:17.052721 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb10fba-ea36-43eb-bda4-a8af12345235 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:17.052815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa31680-e963-48c1-8cb2-41a3f9c61838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:17.054352 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa31680-e963-48c1-8cb2-41a3f9c61838 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:17.060311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb10fba-ea36-43eb-bda4-a8af12345235 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:17.061801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb10fba-ea36-43eb-bda4-a8af12345235 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-bfb10fba-ea36-43eb-bda4-a8af12345235 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:17.082052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:17.083254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:17.084186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:17.084186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:17.084186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:17.084186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:17.084186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:17.084186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:17.084186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:17.084186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:17.084186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:17.084186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:17.087947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 408/813] 10.222.0.42 () {56 vars in 1117 bytes} [Wed May 25 16:07:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:17.088733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb10fba-ea36-43eb-bda4-a8af12345235 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:17.155954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:17.170551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 406/814] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:07:17.171849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2671f4-3d4f-4788-8058-ee44cd8591b5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:17.172136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2671f4-3d4f-4788-8058-ee44cd8591b5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:17.172410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2671f4-3d4f-4788-8058-ee44cd8591b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:17.189459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bb2671f4-3d4f-4788-8058-ee44cd8591b5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:17.190701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bb2671f4-3d4f-4788-8058-ee44cd8591b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:17.200333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:17.202814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb10fba-ea36-43eb-bda4-a8af12345235 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:17.241043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1007a9-6167-4432-9e6c-9b56c572c450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:17.241743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1007a9-6167-4432-9e6c-9b56c572c450 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:17.242023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1007a9-6167-4432-9e6c-9b56c572c450 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:17.242305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1007a9-6167-4432-9e6c-9b56c572c450 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:17.243003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1007a9-6167-4432-9e6c-9b56c572c450 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:17.243576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1007a9-6167-4432-9e6c-9b56c572c450 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:17.244674 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1007a9-6167-4432-9e6c-9b56c572c450 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:17.251321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 409/815] 10.222.0.42 () {56 vars in 1124 bytes} [Wed May 25 16:07:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:17.251962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1007a9-6167-4432-9e6c-9b56c572c450 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:17.282011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 407/816] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:17.282804 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2671f4-3d4f-4788-8058-ee44cd8591b5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:17.368182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1007a9-6167-4432-9e6c-9b56c572c450 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:17.397991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:17.407008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a9f7b-7c51-4029-859a-be19595176d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:17.407620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a9f7b-7c51-4029-859a-be19595176d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:17.407905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a9f7b-7c51-4029-859a-be19595176d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:17.408179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a9f7b-7c51-4029-859a-be19595176d1 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:17.409273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a9f7b-7c51-4029-859a-be19595176d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:17.409467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2671f4-3d4f-4788-8058-ee44cd8591b5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:17.409570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a9f7b-7c51-4029-859a-be19595176d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:17.411051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a9f7b-7c51-4029-859a-be19595176d1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:17.434505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 410/817] 10.222.0.42 () {56 vars in 1149 bytes} [Wed May 25 16:07:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:17.435338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a9f7b-7c51-4029-859a-be19595176d1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:17.453950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f09b50-6962-486e-ad2d-ea50d65693bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:17.454787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f09b50-6962-486e-ad2d-ea50d65693bf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:17.455036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f09b50-6962-486e-ad2d-ea50d65693bf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:17.455308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f09b50-6962-486e-ad2d-ea50d65693bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:17.466689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f09b50-6962-486e-ad2d-ea50d65693bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:17.468128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f09b50-6962-486e-ad2d-ea50d65693bf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:17.470055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a7f09b50-6962-486e-ad2d-ea50d65693bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:17.487183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:17.488281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:17.489172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:17.489172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:17.489172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:17.489172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:17.489172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 408/818] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:17.489447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f09b50-6962-486e-ad2d-ea50d65693bf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:17.552527 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a9f7b-7c51-4029-859a-be19595176d1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:17.593602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09459850-5ec9-44da-a9b0-48ea0e3eec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:17.594375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09459850-5ec9-44da-a9b0-48ea0e3eec84 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:17.594620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09459850-5ec9-44da-a9b0-48ea0e3eec84 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:17.594845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09459850-5ec9-44da-a9b0-48ea0e3eec84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:17.595579 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09459850-5ec9-44da-a9b0-48ea0e3eec84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:17.596941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09459850-5ec9-44da-a9b0-48ea0e3eec84 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:17.599516 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f09b50-6962-486e-ad2d-ea50d65693bf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:17.627986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 411/819] 10.222.0.42 () {60 vars in 1144 bytes} [Wed May 25 16:07:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:17.627986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09459850-5ec9-44da-a9b0-48ea0e3eec84 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:17.654101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ba870-5614-4038-8a7e-93bd11c15c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:17.655464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ba870-5614-4038-8a7e-93bd11c15c2b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:17.655757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ba870-5614-4038-8a7e-93bd11c15c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:17.656035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ba870-5614-4038-8a7e-93bd11c15c2b None admin] PATH_INFO: `/v3/services/network` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:17.656729 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ba870-5614-4038-8a7e-93bd11c15c2b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:17.659114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ba870-5614-4038-8a7e-93bd11c15c2b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ac2ba870-5614-4038-8a7e-93bd11c15c2b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:17.680761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:17.681941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:17.682761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 409/820] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:17 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:17.685027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ba870-5614-4038-8a7e-93bd11c15c2b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:17.718130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:17.772320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:17.775997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09459850-5ec9-44da-a9b0-48ea0e3eec84 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:17.827613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2c80d-c4f1-45e7-86a8-646ab0f4bb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:17.829518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2c80d-c4f1-45e7-86a8-646ab0f4bb5d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:17.830063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2c80d-c4f1-45e7-86a8-646ab0f4bb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:17.830550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2c80d-c4f1-45e7-86a8-646ab0f4bb5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:17.831227 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ba870-5614-4038-8a7e-93bd11c15c2b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:17.850097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2c80d-c4f1-45e7-86a8-646ab0f4bb5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:17.851562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2c80d-c4f1-45e7-86a8-646ab0f4bb5d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-aec2c80d-c4f1-45e7-86a8-646ab0f4bb5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:17.871915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:17.872878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:17.873819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:17.873819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:17.873819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:17.873819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:17.873819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 412/821] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:17.874676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2c80d-c4f1-45e7-86a8-646ab0f4bb5d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:17.884809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bb708-73f4-4102-af8c-e205430a3cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:17.885493 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb708-73f4-4102-af8c-e205430a3cfb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:17.887633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb708-73f4-4102-af8c-e205430a3cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:17.887633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb708-73f4-4102-af8c-e205430a3cfb None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:17.887633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bb708-73f4-4102-af8c-e205430a3cfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:17.888317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bb708-73f4-4102-af8c-e205430a3cfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:17.889396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bb708-73f4-4102-af8c-e205430a3cfb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:17.904788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 410/822] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:17.905647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bb708-73f4-4102-af8c-e205430a3cfb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:17.986187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:17.993212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2c80d-c4f1-45e7-86a8-646ab0f4bb5d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:18.031611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e60f3-b306-408c-b1ab-d22da00ad96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:18.032465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e60f3-b306-408c-b1ab-d22da00ad96a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.032846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e60f3-b306-408c-b1ab-d22da00ad96a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.033221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e60f3-b306-408c-b1ab-d22da00ad96a None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.034965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e60f3-b306-408c-b1ab-d22da00ad96a None admin] RBAC: Adding query filter params (name=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:18.035616 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e60f3-b306-408c-b1ab-d22da00ad96a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:18.039523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e60f3-b306-408c-b1ab-d22da00ad96a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:18.050576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bb708-73f4-4102-af8c-e205430a3cfb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:18.053784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 413/823] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:18.054872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e60f3-b306-408c-b1ab-d22da00ad96a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:18.071560 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:18.098409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475849c4-0fba-441b-9d48-ef9df5dfc3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:18.099440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475849c4-0fba-441b-9d48-ef9df5dfc3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.099955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475849c4-0fba-441b-9d48-ef9df5dfc3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.100450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475849c4-0fba-441b-9d48-ef9df5dfc3a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.101797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475849c4-0fba-441b-9d48-ef9df5dfc3a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:18.102311 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475849c4-0fba-441b-9d48-ef9df5dfc3a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:18.103893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475849c4-0fba-441b-9d48-ef9df5dfc3a1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:18.117315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 411/824] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:18.118524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475849c4-0fba-441b-9d48-ef9df5dfc3a1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:18.175177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e60f3-b306-408c-b1ab-d22da00ad96a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:18.213798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d74cb4-8338-48df-af62-f31f1640ab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:18.215581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d74cb4-8338-48df-af62-f31f1640ab14 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.215929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d74cb4-8338-48df-af62-f31f1640ab14 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.216203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d74cb4-8338-48df-af62-f31f1640ab14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.223942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d74cb4-8338-48df-af62-f31f1640ab14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:18.225402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d74cb4-8338-48df-af62-f31f1640ab14 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:18.232610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475849c4-0fba-441b-9d48-ef9df5dfc3a1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-69d74cb4-8338-48df-af62-f31f1640ab14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:18.240413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:18.241303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:18.242452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:18.242452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:18.242452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:18.242452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:18.242452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:18.242452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:18.242452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:18.242452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:18.242452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 414/825] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:18.243118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d74cb4-8338-48df-af62-f31f1640ab14 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:18.263412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:18.280141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd890b3-51cf-411c-9a03-c493bdde2041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:18.280821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd890b3-51cf-411c-9a03-c493bdde2041 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.281109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd890b3-51cf-411c-9a03-c493bdde2041 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.281634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd890b3-51cf-411c-9a03-c493bdde2041 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.282426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd890b3-51cf-411c-9a03-c493bdde2041 None admin] RBAC: Adding query filter params (type=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:18.283887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd890b3-51cf-411c-9a03-c493bdde2041 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:18.285279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd890b3-51cf-411c-9a03-c493bdde2041 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:18.293522 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 412/826] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:18.294700 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd890b3-51cf-411c-9a03-c493bdde2041 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:18.349614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:18.358602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d74cb4-8338-48df-af62-f31f1640ab14 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:18.398578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f3c4a-a5ce-481d-9e4b-c5f7b01284e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:18.399503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f3c4a-a5ce-481d-9e4b-c5f7b01284e7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.399864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f3c4a-a5ce-481d-9e4b-c5f7b01284e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.400215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f3c4a-a5ce-481d-9e4b-c5f7b01284e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.410008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f3c4a-a5ce-481d-9e4b-c5f7b01284e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:18.412136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f3c4a-a5ce-481d-9e4b-c5f7b01284e7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e56f3c4a-a5ce-481d-9e4b-c5f7b01284e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:18.431669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:18.433522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:18.435440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:18.435440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:18.435440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:18.435440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:18.435440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:18.435440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:18.435440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:18.435440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:18.438460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 415/827] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:18.439944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f3c4a-a5ce-481d-9e4b-c5f7b01284e7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:18.447158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd890b3-51cf-411c-9a03-c493bdde2041 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:18.493970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333ad05-3c3a-43d1-99f0-0119e0287c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:18.494604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333ad05-3c3a-43d1-99f0-0119e0287c1a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.494935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333ad05-3c3a-43d1-99f0-0119e0287c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.495224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333ad05-3c3a-43d1-99f0-0119e0287c1a None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.496656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333ad05-3c3a-43d1-99f0-0119e0287c1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:18.496974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333ad05-3c3a-43d1-99f0-0119e0287c1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:18.498949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333ad05-3c3a-43d1-99f0-0119e0287c1a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:18.516422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 413/828] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:18.517445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333ad05-3c3a-43d1-99f0-0119e0287c1a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:18.545195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:18.571088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f3c4a-a5ce-481d-9e4b-c5f7b01284e7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:18.609943 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f52219-5314-476c-8612-d89c02f86c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:18.610615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f52219-5314-476c-8612-d89c02f86c5f None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.610899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f52219-5314-476c-8612-d89c02f86c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.611537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f52219-5314-476c-8612-d89c02f86c5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.612017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f52219-5314-476c-8612-d89c02f86c5f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:18.613398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f52219-5314-476c-8612-d89c02f86c5f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:18.628212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:18.633904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333ad05-3c3a-43d1-99f0-0119e0287c1a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:18.639167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 416/829] 10.222.0.42 () {60 vars in 1144 bytes} [Wed May 25 16:07:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:18.640107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f52219-5314-476c-8612-d89c02f86c5f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:18.690626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b383c9-b468-416f-884f-9de505a4e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:18.691707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b383c9-b468-416f-884f-9de505a4e056 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.692217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b383c9-b468-416f-884f-9de505a4e056 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.692691 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b383c9-b468-416f-884f-9de505a4e056 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.693670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b383c9-b468-416f-884f-9de505a4e056 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:18.694316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b383c9-b468-416f-884f-9de505a4e056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:18.695958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b383c9-b468-416f-884f-9de505a4e056 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:18.711262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 414/830] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:18.717705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b383c9-b468-416f-884f-9de505a4e056 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:18.762061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f52219-5314-476c-8612-d89c02f86c5f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:18.801200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f1578-3a9f-4dad-b83e-49ce19e2906e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:18.801900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f1578-3a9f-4dad-b83e-49ce19e2906e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.802198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f1578-3a9f-4dad-b83e-49ce19e2906e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.802867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f1578-3a9f-4dad-b83e-49ce19e2906e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.803389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f1578-3a9f-4dad-b83e-49ce19e2906e None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=6e6795b5b5cf4499ad494b875756ba71) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:18.803708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f1578-3a9f-4dad-b83e-49ce19e2906e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:18.805261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f1578-3a9f-4dad-b83e-49ce19e2906e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:18.807446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:18.819492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 417/831] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:18 2022] GET /identity/v3/role_assignments?user.id=6e6795b5b5cf4499ad494b875756ba71&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:18.848433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b383c9-b468-416f-884f-9de505a4e056 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:18.861640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed974d-8094-4778-a61c-93d3350805af None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.861991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed974d-8094-4778-a61c-93d3350805af None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.862241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed974d-8094-4778-a61c-93d3350805af None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.863698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 418/832] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:07:18.871920 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048c7a-875b-4209-a690-6e2228f73278 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.872302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048c7a-875b-4209-a690-6e2228f73278 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.872591 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048c7a-875b-4209-a690-6e2228f73278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.893509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6dc0-22e9-4b02-9f8c-b6c97316d8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:18.894906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6dc0-22e9-4b02-9f8c-b6c97316d8f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:18.895402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6dc0-22e9-4b02-9f8c-b6c97316d8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:18.895886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6dc0-22e9-4b02-9f8c-b6c97316d8f5 None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/730cb89f7a214fde9c5040674323f8db/roles/6bc01dbf83c844fe802466d3c4ada6af` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:18.911022 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1f048c7a-875b-4209-a690-6e2228f73278 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:18.912331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1f048c7a-875b-4209-a690-6e2228f73278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:18.916062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f6dc0-22e9-4b02-9f8c-b6c97316d8f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=730cb89f7a214fde9c5040674323f8db, role_id=6bc01dbf83c844fe802466d3c4ada6af)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:18.918151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f6dc0-22e9-4b02-9f8c-b6c97316d8f5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:18.936507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 415/833] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:07:18 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/730cb89f7a214fde9c5040674323f8db/roles/6bc01dbf83c844fe802466d3c4ada6af => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:07:19.022292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 419/834] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:19.047558 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6dc0-22e9-4b02-9f8c-b6c97316d8f5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:19.054144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:19.205913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6dc0-22e9-4b02-9f8c-b6c97316d8f5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:19.247404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd26bb-fee4-4da4-b83d-411cf5e9761f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:19.248210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd26bb-fee4-4da4-b83d-411cf5e9761f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:19.248494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd26bb-fee4-4da4-b83d-411cf5e9761f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:19.248763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd26bb-fee4-4da4-b83d-411cf5e9761f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:19.259767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd26bb-fee4-4da4-b83d-411cf5e9761f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-94bd26bb-fee4-4da4-b83d-411cf5e9761f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:19.267818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:19.268855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:19.269758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 416/835] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:19.277174 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f048c7a-875b-4209-a690-6e2228f73278 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:19.390630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f048c7a-875b-4209-a690-6e2228f73278 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:19.419479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:19.432514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf21551-e6a7-4cfe-abb3-ef61110b0ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:19.434524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf21551-e6a7-4cfe-abb3-ef61110b0ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:19.434524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf21551-e6a7-4cfe-abb3-ef61110b0ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:19.434524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf21551-e6a7-4cfe-abb3-ef61110b0ac0 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:19.434783 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf21551-e6a7-4cfe-abb3-ef61110b0ac0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:19.434982 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf21551-e6a7-4cfe-abb3-ef61110b0ac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:19.436334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf21551-e6a7-4cfe-abb3-ef61110b0ac0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:19.445739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 420/836] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:07:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:19.452051 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd26bb-fee4-4da4-b83d-411cf5e9761f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:19.538193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:19.563492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd26bb-fee4-4da4-b83d-411cf5e9761f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:19.599861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bfcb0-cdda-4c8f-91c4-a492ebe2f6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:19.600677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bfcb0-cdda-4c8f-91c4-a492ebe2f6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:19.600958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bfcb0-cdda-4c8f-91c4-a492ebe2f6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:19.601232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bfcb0-cdda-4c8f-91c4-a492ebe2f6d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:19.613190 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bfcb0-cdda-4c8f-91c4-a492ebe2f6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:19.614747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bfcb0-cdda-4c8f-91c4-a492ebe2f6d4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f75bfcb0-cdda-4c8f-91c4-a492ebe2f6d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:19.633061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:19.634473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:19.635353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:19.635353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:19.635353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:19.635353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:19.635353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 417/837] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:19.640735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf21551-e6a7-4cfe-abb3-ef61110b0ac0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:19.748233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf21551-e6a7-4cfe-abb3-ef61110b0ac0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:19.784699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19195a50-e232-40e9-af48-61c32d8ec2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:19.785372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19195a50-e232-40e9-af48-61c32d8ec2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:19.785653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19195a50-e232-40e9-af48-61c32d8ec2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:19.785955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19195a50-e232-40e9-af48-61c32d8ec2d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:19.786683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19195a50-e232-40e9-af48-61c32d8ec2d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:19.786991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19195a50-e232-40e9-af48-61c32d8ec2d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:19.788418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19195a50-e232-40e9-af48-61c32d8ec2d2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:19.805558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 421/838] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:19.810842 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bfcb0-cdda-4c8f-91c4-a492ebe2f6d4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:19.920808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bfcb0-cdda-4c8f-91c4-a492ebe2f6d4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:19.938127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:19.958068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92c1ef-18d2-4f66-a4ae-0e1fe404231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:19.959596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92c1ef-18d2-4f66-a4ae-0e1fe404231e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:19.960088 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92c1ef-18d2-4f66-a4ae-0e1fe404231e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:19.960576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92c1ef-18d2-4f66-a4ae-0e1fe404231e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:19.969967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92c1ef-18d2-4f66-a4ae-0e1fe404231e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:19.971688 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92c1ef-18d2-4f66-a4ae-0e1fe404231e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4c92c1ef-18d2-4f66-a4ae-0e1fe404231e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:19.986736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:19.987761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:19.988707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:19.988707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:19.988707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:19.988707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:19.988707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:19.988707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:19.988707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:19.988707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:19.988707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 16:07:19.988707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:19.991569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 418/839] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:20.001736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19195a50-e232-40e9-af48-61c32d8ec2d2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:20.050180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:20.104452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19195a50-e232-40e9-af48-61c32d8ec2d2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:20.144478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0989d-8b27-4012-b596-60035995f179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:20.145157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0989d-8b27-4012-b596-60035995f179 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:20.145440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0989d-8b27-4012-b596-60035995f179 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:20.145742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0989d-8b27-4012-b596-60035995f179 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:20.146588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0989d-8b27-4012-b596-60035995f179 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:20.146899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0989d-8b27-4012-b596-60035995f179 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:20.148340 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0989d-8b27-4012-b596-60035995f179 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:20.171051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 422/840] 10.222.0.42 () {56 vars in 1135 bytes} [Wed May 25 16:07:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:20.176626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92c1ef-18d2-4f66-a4ae-0e1fe404231e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:20.286356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92c1ef-18d2-4f66-a4ae-0e1fe404231e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:20.311451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cd3f7-1d3a-4ca3-b6ec-d09b91226257 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:20.311798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cd3f7-1d3a-4ca3-b6ec-d09b91226257 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:20.312003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cd3f7-1d3a-4ca3-b6ec-d09b91226257 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:20.314770 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 423/841] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:20.319255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83681-a0e5-46b0-b093-2a97314b5774 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:20.319417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83681-a0e5-46b0-b093-2a97314b5774 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:20.319627 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83681-a0e5-46b0-b093-2a97314b5774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:20.324689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911ce7d-4b8b-43f9-bdac-2de938035dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:20.325637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911ce7d-4b8b-43f9-bdac-2de938035dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:20.326189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911ce7d-4b8b-43f9-bdac-2de938035dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:20.326682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911ce7d-4b8b-43f9-bdac-2de938035dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:20.337394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8ed83681-a0e5-46b0-b093-2a97314b5774 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:20.338659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8ed83681-a0e5-46b0-b093-2a97314b5774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:20.376749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:20.402253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911ce7d-4b8b-43f9-bdac-2de938035dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:20.404084 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911ce7d-4b8b-43f9-bdac-2de938035dc6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:20.434728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 424/842] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:20.445598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed83681-a0e5-46b0-b093-2a97314b5774 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:20.488772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:20.521195 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 419/843] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:20 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:07:20.528406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911ce7d-4b8b-43f9-bdac-2de938035dc6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:20.557408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed83681-a0e5-46b0-b093-2a97314b5774 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:20.599888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7250c-1186-484d-b655-81debe3666be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:20.600611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7250c-1186-484d-b655-81debe3666be None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:20.600889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7250c-1186-484d-b655-81debe3666be None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:20.601161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7250c-1186-484d-b655-81debe3666be None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:20.601850 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7250c-1186-484d-b655-81debe3666be None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:20.603959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7250c-1186-484d-b655-81debe3666be None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:20.611320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-97e7250c-1186-484d-b655-81debe3666be None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:20.617649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:20.618837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 16:07:20.619675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:20.628424 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 425/844] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:07:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:20.630114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7250c-1186-484d-b655-81debe3666be None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:20.642442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911ce7d-4b8b-43f9-bdac-2de938035dc6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:20.679273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f7c38-9498-49ac-aca0-d9aa2ff79433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:20.680076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f7c38-9498-49ac-aca0-d9aa2ff79433 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:20.680359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f7c38-9498-49ac-aca0-d9aa2ff79433 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:20.680633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f7c38-9498-49ac-aca0-d9aa2ff79433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:20.691540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f7c38-9498-49ac-aca0-d9aa2ff79433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:20.692990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f7c38-9498-49ac-aca0-d9aa2ff79433 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-058f7c38-9498-49ac-aca0-d9aa2ff79433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:20.711750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:20.712723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:20.714517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:20.714517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:20.714517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:20.714517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:20.718572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 420/845] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:20.718832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f7c38-9498-49ac-aca0-d9aa2ff79433 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:20.752521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7250c-1186-484d-b655-81debe3666be None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:20.755050 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:20.797817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a79465-6a13-4599-811b-d1a2813ad34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:20.798927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a79465-6a13-4599-811b-d1a2813ad34b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:20.799430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a79465-6a13-4599-811b-d1a2813ad34b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:20.799942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a79465-6a13-4599-811b-d1a2813ad34b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:20.800958 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a79465-6a13-4599-811b-d1a2813ad34b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:20.801490 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a79465-6a13-4599-811b-d1a2813ad34b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:20.803200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a79465-6a13-4599-811b-d1a2813ad34b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:20.814465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 426/846] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:07:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:20.816261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a79465-6a13-4599-811b-d1a2813ad34b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:20.827210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f7c38-9498-49ac-aca0-d9aa2ff79433 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:20.854418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:20.863471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ab816-19a5-4b7a-ab82-26528517f198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:20.864151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ab816-19a5-4b7a-ab82-26528517f198 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:20.864438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ab816-19a5-4b7a-ab82-26528517f198 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:20.864715 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ab816-19a5-4b7a-ab82-26528517f198 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:20.865796 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ab816-19a5-4b7a-ab82-26528517f198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:20.865908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ab816-19a5-4b7a-ab82-26528517f198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:20.867220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ab816-19a5-4b7a-ab82-26528517f198 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:20.885010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 421/847] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:20.886770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ab816-19a5-4b7a-ab82-26528517f198 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:20.934579 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a79465-6a13-4599-811b-d1a2813ad34b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:20.972501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8be9a-c9f1-4012-be83-947cb2255e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:20.973591 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8be9a-c9f1-4012-be83-947cb2255e23 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:20.974127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8be9a-c9f1-4012-be83-947cb2255e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:20.974662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8be9a-c9f1-4012-be83-947cb2255e23 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:20.975570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8be9a-c9f1-4012-be83-947cb2255e23 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:20.976078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8be9a-c9f1-4012-be83-947cb2255e23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:20.977665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8be9a-c9f1-4012-be83-947cb2255e23 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:20.989854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 427/848] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:07:20 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:20.996115 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ab816-19a5-4b7a-ab82-26528517f198 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:21.039292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce7774-a4d8-4dcb-a3ac-51e4b89dfa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:21.040107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce7774-a4d8-4dcb-a3ac-51e4b89dfa41 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:21.040394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce7774-a4d8-4dcb-a3ac-51e4b89dfa41 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:21.040672 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce7774-a4d8-4dcb-a3ac-51e4b89dfa41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:21.048939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce7774-a4d8-4dcb-a3ac-51e4b89dfa41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:21.050524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce7774-a4d8-4dcb-a3ac-51e4b89dfa41 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7dce7774-a4d8-4dcb-a3ac-51e4b89dfa41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:21.067353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:21.068327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:21.069867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:21.069867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:21.069867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:21.069867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:21.069867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:21.069867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:21.069867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:21.069867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:21.069867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 422/849] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:21.075052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8be9a-c9f1-4012-be83-947cb2255e23 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:21.111852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:21.204050 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8be9a-c9f1-4012-be83-947cb2255e23 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:21.241102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef961c-6ac0-4e04-bf21-479cba440e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:21.241791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef961c-6ac0-4e04-bf21-479cba440e6b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:21.242077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef961c-6ac0-4e04-bf21-479cba440e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:21.242350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef961c-6ac0-4e04-bf21-479cba440e6b None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:21.243141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef961c-6ac0-4e04-bf21-479cba440e6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:21.243446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef961c-6ac0-4e04-bf21-479cba440e6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:21.244876 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef961c-6ac0-4e04-bf21-479cba440e6b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:21.259020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 428/850] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:21.264740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce7774-a4d8-4dcb-a3ac-51e4b89dfa41 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:21.323154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:21.353159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe972-ce67-4164-947f-9f1ea85fdcb1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:21.353445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe972-ce67-4164-947f-9f1ea85fdcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:21.353738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe972-ce67-4164-947f-9f1ea85fdcb1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:21.356183 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 429/851] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:21.362508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df0df-1bad-4304-bc96-50c361762985 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:21.362759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df0df-1bad-4304-bc96-50c361762985 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:21.362995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df0df-1bad-4304-bc96-50c361762985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:21.380892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce7774-a4d8-4dcb-a3ac-51e4b89dfa41 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:21.382708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4d9df0df-1bad-4304-bc96-50c361762985 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:21.384742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4d9df0df-1bad-4304-bc96-50c361762985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:21.422196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e51785-691d-4218-894b-8a354c15e27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:21.422861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51785-691d-4218-894b-8a354c15e27d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:21.423267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51785-691d-4218-894b-8a354c15e27d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:21.423460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51785-691d-4218-894b-8a354c15e27d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:21.424326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e51785-691d-4218-894b-8a354c15e27d None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=c6eb13611d0d4b6abe035c9bfbfde79c) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:21.424554 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e51785-691d-4218-894b-8a354c15e27d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:21.426018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e51785-691d-4218-894b-8a354c15e27d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:21.435767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 423/852] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:21 2022] GET /identity/v3/role_assignments?user.id=c6eb13611d0d4b6abe035c9bfbfde79c&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:21.437105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f9a00-4774-4ceb-a87e-2e6464c9fc1b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:21.437400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f9a00-4774-4ceb-a87e-2e6464c9fc1b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:21.437681 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f9a00-4774-4ceb-a87e-2e6464c9fc1b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:21.441963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 424/853] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:21.445045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c6e29-df95-4945-afb4-53788ce181ec None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:21.445333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c6e29-df95-4945-afb4-53788ce181ec None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:21.445613 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c6e29-df95-4945-afb4-53788ce181ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:21.468889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-be5c6e29-df95-4945-afb4-53788ce181ec None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:21.470184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-be5c6e29-df95-4945-afb4-53788ce181ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:21.487082 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 430/854] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:21.510440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9df0df-1bad-4304-bc96-50c361762985 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:21.543120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:21.569763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 425/855] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:21.592726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c6e29-df95-4945-afb4-53788ce181ec None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:21.646710 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:21.647307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9df0df-1bad-4304-bc96-50c361762985 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:21.694032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888514be-d6a2-46a9-902e-bd804d17c3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:21.694985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888514be-d6a2-46a9-902e-bd804d17c3ea None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:21.695284 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888514be-d6a2-46a9-902e-bd804d17c3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:21.695552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888514be-d6a2-46a9-902e-bd804d17c3ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:21.707889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888514be-d6a2-46a9-902e-bd804d17c3ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-888514be-d6a2-46a9-902e-bd804d17c3ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:21.716286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:21.717240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:21.718150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 431/856] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:07:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:21.723472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888514be-d6a2-46a9-902e-bd804d17c3ea None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:21.729087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c6e29-df95-4945-afb4-53788ce181ec None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:21.773310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f750ba0-daef-4089-af9b-655d92d0efda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:21.774197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f750ba0-daef-4089-af9b-655d92d0efda None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:21.774580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f750ba0-daef-4089-af9b-655d92d0efda None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:21.774953 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f750ba0-daef-4089-af9b-655d92d0efda None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:21.775713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f750ba0-daef-4089-af9b-655d92d0efda None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:21.778367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f750ba0-daef-4089-af9b-655d92d0efda None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9f750ba0-daef-4089-af9b-655d92d0efda None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:21.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:21.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 16:07:21.794507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:21.796766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 426/857] 10.222.0.42 () {56 vars in 1132 bytes} [Wed May 25 16:07:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:21.802279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f750ba0-daef-4089-af9b-655d92d0efda None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:21.831565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:21.854857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888514be-d6a2-46a9-902e-bd804d17c3ea None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:21.894754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2fb9a-0573-4087-a561-3b2bffef1826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:21.895700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2fb9a-0573-4087-a561-3b2bffef1826 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:21.896151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2fb9a-0573-4087-a561-3b2bffef1826 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:21.896590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2fb9a-0573-4087-a561-3b2bffef1826 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:21.897599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2fb9a-0573-4087-a561-3b2bffef1826 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:21.898237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2fb9a-0573-4087-a561-3b2bffef1826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:21.899792 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2fb9a-0573-4087-a561-3b2bffef1826 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:21.910654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 432/858] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:07:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:21.911783 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f750ba0-daef-4089-af9b-655d92d0efda None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:21.919460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2fb9a-0573-4087-a561-3b2bffef1826 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:21.920046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:21.962043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5e972-d3d0-4784-b10c-8eb68df16c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:21.965931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5e972-d3d0-4784-b10c-8eb68df16c2f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:21.966454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5e972-d3d0-4784-b10c-8eb68df16c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:21.966932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5e972-d3d0-4784-b10c-8eb68df16c2f None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:21.968475 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5e972-d3d0-4784-b10c-8eb68df16c2f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:21.969003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5e972-d3d0-4784-b10c-8eb68df16c2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:21.971456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5e972-d3d0-4784-b10c-8eb68df16c2f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:21.978856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 427/859] 10.222.0.42 () {56 vars in 1126 bytes} [Wed May 25 16:07:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:21.984923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5e972-d3d0-4784-b10c-8eb68df16c2f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:22.035964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2fb9a-0573-4087-a561-3b2bffef1826 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:22.072123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7111d-fb27-4645-8684-68961ceaade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:22.072923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7111d-fb27-4645-8684-68961ceaade3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:22.073212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7111d-fb27-4645-8684-68961ceaade3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:22.073515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7111d-fb27-4645-8684-68961ceaade3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:22.084904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7111d-fb27-4645-8684-68961ceaade3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:22.086405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7111d-fb27-4645-8684-68961ceaade3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:22.088250 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:22.095884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5e972-d3d0-4784-b10c-8eb68df16c2f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d6a7111d-fb27-4645-8684-68961ceaade3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:22.107566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:22.108502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:22.109399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:22.109399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:22.109399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:22.109399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:22.110379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 433/860] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:22.114895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7111d-fb27-4645-8684-68961ceaade3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:22.143090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b985d6-a84a-4457-8230-1c4e6c1249fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:22.143791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b985d6-a84a-4457-8230-1c4e6c1249fb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:22.144078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b985d6-a84a-4457-8230-1c4e6c1249fb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:22.144357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b985d6-a84a-4457-8230-1c4e6c1249fb None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:22.145518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b985d6-a84a-4457-8230-1c4e6c1249fb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:22.146963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b985d6-a84a-4457-8230-1c4e6c1249fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:22.148708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b985d6-a84a-4457-8230-1c4e6c1249fb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:22.161741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 428/861] 10.222.0.42 () {56 vars in 1126 bytes} [Wed May 25 16:07:21 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:22.243426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7111d-fb27-4645-8684-68961ceaade3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:22.282551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a63b11-b10b-4759-8396-9547219f2fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:22.283228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a63b11-b10b-4759-8396-9547219f2fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:22.283517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a63b11-b10b-4759-8396-9547219f2fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:22.284050 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a63b11-b10b-4759-8396-9547219f2fb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:22.284791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a63b11-b10b-4759-8396-9547219f2fb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:22.285100 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a63b11-b10b-4759-8396-9547219f2fb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:22.286564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a63b11-b10b-4759-8396-9547219f2fb5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:22.300981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 434/862] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:22.306670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b985d6-a84a-4457-8230-1c4e6c1249fb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:22.346797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:22.428012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b985d6-a84a-4457-8230-1c4e6c1249fb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:22.462769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b689e-3409-40f1-a987-f05ed62c0a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:22.463596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b689e-3409-40f1-a987-f05ed62c0a54 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:22.463881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b689e-3409-40f1-a987-f05ed62c0a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:22.464756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b689e-3409-40f1-a987-f05ed62c0a54 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:22.472870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b689e-3409-40f1-a987-f05ed62c0a54 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:22.474405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b689e-3409-40f1-a987-f05ed62c0a54 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ef4b689e-3409-40f1-a987-f05ed62c0a54 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:22.489843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:22.490829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:22.491760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:22.491760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:22.491760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:22.491760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:22.491760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:22.491760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:22.491760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:22.491760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:22.491760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:22.491760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:22.491760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 429/863] 10.222.0.42 () {56 vars in 1117 bytes} [Wed May 25 16:07:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:22.497761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a63b11-b10b-4759-8396-9547219f2fb5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:22.556680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:22.605836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a63b11-b10b-4759-8396-9547219f2fb5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:22.642541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2074f-943a-4dd8-ba6c-b5431a0d7375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:22.643247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2074f-943a-4dd8-ba6c-b5431a0d7375 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:22.643531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2074f-943a-4dd8-ba6c-b5431a0d7375 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:22.643808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2074f-943a-4dd8-ba6c-b5431a0d7375 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:22.644679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2074f-943a-4dd8-ba6c-b5431a0d7375 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:22.644995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2074f-943a-4dd8-ba6c-b5431a0d7375 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:22.646513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2074f-943a-4dd8-ba6c-b5431a0d7375 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:22.667006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 435/864] 10.222.0.42 () {56 vars in 1149 bytes} [Wed May 25 16:07:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:22.672692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b689e-3409-40f1-a987-f05ed62c0a54 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:22.775785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:22.788055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b689e-3409-40f1-a987-f05ed62c0a54 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:22.826843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a391429-83e6-47a6-9879-07301ea7fb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:22.827911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391429-83e6-47a6-9879-07301ea7fb94 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:22.828424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391429-83e6-47a6-9879-07301ea7fb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:22.828908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391429-83e6-47a6-9879-07301ea7fb94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:22.902634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a391429-83e6-47a6-9879-07301ea7fb94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:22.904442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a391429-83e6-47a6-9879-07301ea7fb94 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:23.009635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:23.018902 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 430/865] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:22 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:07:23.029131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2074f-943a-4dd8-ba6c-b5431a0d7375 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:23.140909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2074f-943a-4dd8-ba6c-b5431a0d7375 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:23.155241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:23.178143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664864d-3e08-44da-8d0f-56756b873cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:23.178945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664864d-3e08-44da-8d0f-56756b873cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:23.179231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664864d-3e08-44da-8d0f-56756b873cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:23.179510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664864d-3e08-44da-8d0f-56756b873cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:23.191595 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664864d-3e08-44da-8d0f-56756b873cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:23.193094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664864d-3e08-44da-8d0f-56756b873cd7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c664864d-3e08-44da-8d0f-56756b873cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:23.214150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:23.215299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:23.216198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:23.216198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:23.216198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:23.216198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:23.216198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 436/866] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:23.221568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a391429-83e6-47a6-9879-07301ea7fb94 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:23.325594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a391429-83e6-47a6-9879-07301ea7fb94 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:23.365835 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b80290-b0a1-49c5-928d-222d8902a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:23.366793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b80290-b0a1-49c5-928d-222d8902a210 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:23.367082 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b80290-b0a1-49c5-928d-222d8902a210 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:23.367422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b80290-b0a1-49c5-928d-222d8902a210 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:23.368158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b80290-b0a1-49c5-928d-222d8902a210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:23.368471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b80290-b0a1-49c5-928d-222d8902a210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:23.369870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b80290-b0a1-49c5-928d-222d8902a210 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:23.382605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 431/867] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:23.388044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664864d-3e08-44da-8d0f-56756b873cd7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:23.395316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b57864-5496-4812-8a6c-022112c80450 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:23.395602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b57864-5496-4812-8a6c-022112c80450 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:23.395886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b57864-5496-4812-8a6c-022112c80450 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:23.397540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 432/868] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:23.403857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1f3f-f4c1-4271-b52e-fd776da8212f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:23.404345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1f3f-f4c1-4271-b52e-fd776da8212f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:23.404789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1f3f-f4c1-4271-b52e-fd776da8212f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:23.423155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-970a1f3f-f4c1-4271-b52e-fd776da8212f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:23.424931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-970a1f3f-f4c1-4271-b52e-fd776da8212f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:23.481922 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:23.506304 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664864d-3e08-44da-8d0f-56756b873cd7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:23.507424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 433/869] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:23.531071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a1f3f-f4c1-4271-b52e-fd776da8212f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:23.549502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5585d-72f9-409f-9777-7eea90474a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:23.550343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5585d-72f9-409f-9777-7eea90474a9d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:23.550684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5585d-72f9-409f-9777-7eea90474a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:23.551245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5585d-72f9-409f-9777-7eea90474a9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:23.560750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5585d-72f9-409f-9777-7eea90474a9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:23.562970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5585d-72f9-409f-9777-7eea90474a9d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e7c5585d-72f9-409f-9777-7eea90474a9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:23.581091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:23.582347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:23.583870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:23.583870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:23.583870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:23.583870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:23.583870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:23.583870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:23.583870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:23.583870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:23.585793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 437/870] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:23.588801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5585d-72f9-409f-9777-7eea90474a9d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:23.616568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:23.648154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a1f3f-f4c1-4271-b52e-fd776da8212f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:23.687802 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca67bdf-d891-40d1-a5a2-a4e633f098bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:23.689541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca67bdf-d891-40d1-a5a2-a4e633f098bb None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:23.690192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca67bdf-d891-40d1-a5a2-a4e633f098bb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:23.690691 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca67bdf-d891-40d1-a5a2-a4e633f098bb None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:23.691643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca67bdf-d891-40d1-a5a2-a4e633f098bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:23.693354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca67bdf-d891-40d1-a5a2-a4e633f098bb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:23.707176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5585d-72f9-409f-9777-7eea90474a9d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:23.710575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 434/871] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:07:23 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:07:23.761961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25596fc6-e479-4fe4-8244-89c3a5382d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:23.763087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25596fc6-e479-4fe4-8244-89c3a5382d4b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:23.763618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25596fc6-e479-4fe4-8244-89c3a5382d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:23.764120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25596fc6-e479-4fe4-8244-89c3a5382d4b None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:23.765126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25596fc6-e479-4fe4-8244-89c3a5382d4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:23.765747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25596fc6-e479-4fe4-8244-89c3a5382d4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:23.767614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25596fc6-e479-4fe4-8244-89c3a5382d4b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:23.784230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 438/872] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:23.790763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca67bdf-d891-40d1-a5a2-a4e633f098bb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:23.838108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626d1e6-06a4-4c5d-b2b7-25e4c2eb540a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:23.838376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626d1e6-06a4-4c5d-b2b7-25e4c2eb540a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:23.838599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626d1e6-06a4-4c5d-b2b7-25e4c2eb540a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:23.840812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 439/873] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:23.846445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad0ae3-d949-4d31-9a54-53f962aaed0e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:23.846726 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad0ae3-d949-4d31-9a54-53f962aaed0e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:23.847005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad0ae3-d949-4d31-9a54-53f962aaed0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:23.851076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:23.868450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d0ad0ae3-d949-4d31-9a54-53f962aaed0e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:23.869664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d0ad0ae3-d949-4d31-9a54-53f962aaed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:23.929215 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca67bdf-d891-40d1-a5a2-a4e633f098bb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:23.959676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 440/874] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:23 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:23.971023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948d72d-7a0d-42d5-a83f-6d55c255dfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:23.971676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948d72d-7a0d-42d5-a83f-6d55c255dfb2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:23.971952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948d72d-7a0d-42d5-a83f-6d55c255dfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:23.972233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948d72d-7a0d-42d5-a83f-6d55c255dfb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:23.973067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948d72d-7a0d-42d5-a83f-6d55c255dfb2 None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=730cb89f7a214fde9c5040674323f8db) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:23.973387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948d72d-7a0d-42d5-a83f-6d55c255dfb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:23.974828 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948d72d-7a0d-42d5-a83f-6d55c255dfb2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:23.982586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad0ae3-d949-4d31-9a54-53f962aaed0e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:23.983794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 435/875] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:23 2022] GET /identity/v3/role_assignments?user.id=730cb89f7a214fde9c5040674323f8db&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:23.999818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:24.102104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad0ae3-d949-4d31-9a54-53f962aaed0e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:24.142841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c931b6-098a-4a7c-a0fd-c801b7862896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:24.144083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c931b6-098a-4a7c-a0fd-c801b7862896 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:24.144597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c931b6-098a-4a7c-a0fd-c801b7862896 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:24.145085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c931b6-098a-4a7c-a0fd-c801b7862896 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:24.146018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c931b6-098a-4a7c-a0fd-c801b7862896 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:24.147635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c931b6-098a-4a7c-a0fd-c801b7862896 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-97c931b6-098a-4a7c-a0fd-c801b7862896 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:24.162095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:24.167513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:24.169524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:24.172793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 441/876] 10.222.0.42 () {56 vars in 1126 bytes} [Wed May 25 16:07:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:24.179459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948d72d-7a0d-42d5-a83f-6d55c255dfb2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:24.300501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948d72d-7a0d-42d5-a83f-6d55c255dfb2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:24.312286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:24.342742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f7165-eb32-4d96-9a7c-3bc73aa26978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:24.343423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f7165-eb32-4d96-9a7c-3bc73aa26978 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:24.343706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f7165-eb32-4d96-9a7c-3bc73aa26978 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:24.343980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f7165-eb32-4d96-9a7c-3bc73aa26978 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:24.344690 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f7165-eb32-4d96-9a7c-3bc73aa26978 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:24.344995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f7165-eb32-4d96-9a7c-3bc73aa26978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:24.346418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f7165-eb32-4d96-9a7c-3bc73aa26978 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:24.352075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 436/877] 10.222.0.42 () {56 vars in 1122 bytes} [Wed May 25 16:07:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:24.358302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c931b6-098a-4a7c-a0fd-c801b7862896 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:24.451718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:24.473564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c931b6-098a-4a7c-a0fd-c801b7862896 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:24.513223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24795714-e774-4f7a-9201-b813e7200830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:24.513933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24795714-e774-4f7a-9201-b813e7200830 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:24.514215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24795714-e774-4f7a-9201-b813e7200830 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:24.514491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24795714-e774-4f7a-9201-b813e7200830 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:24.515201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24795714-e774-4f7a-9201-b813e7200830 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:24.515502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24795714-e774-4f7a-9201-b813e7200830 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:24.516881 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24795714-e774-4f7a-9201-b813e7200830 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:24.522164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 442/878] 10.222.0.42 () {56 vars in 1122 bytes} [Wed May 25 16:07:24 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:24.666966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b7eb-fee4-46a7-8824-e1efc85f493b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:24.667765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b7eb-fee4-46a7-8824-e1efc85f493b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:24.668159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b7eb-fee4-46a7-8824-e1efc85f493b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:24.669943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 437/879] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:24.675852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490344f7-f701-4741-b029-cf009bddde2c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:24.676486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490344f7-f701-4741-b029-cf009bddde2c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:24.676976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490344f7-f701-4741-b029-cf009bddde2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:24.695968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-490344f7-f701-4741-b029-cf009bddde2c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:24.697579 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-490344f7-f701-4741-b029-cf009bddde2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:24.775039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 443/880] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:24 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:24.798239 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8b7eb-fee4-46a7-8824-e1efc85f493b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:24.912547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8b7eb-fee4-46a7-8824-e1efc85f493b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:24.948255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb71c9-8fa8-4eb7-8840-514e49cf0df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:24.949803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb71c9-8fa8-4eb7-8840-514e49cf0df2 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:24.950319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb71c9-8fa8-4eb7-8840-514e49cf0df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:24.950808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb71c9-8fa8-4eb7-8840-514e49cf0df2 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:24.951749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb71c9-8fa8-4eb7-8840-514e49cf0df2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:24.953376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb71c9-8fa8-4eb7-8840-514e49cf0df2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:24.977864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 438/881] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:07:24 2022] POST /identity/v3/services => generated 269 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:26.132960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50c280-f60d-4ada-8dbb-57fc06c2ffcc None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:26.134036 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50c280-f60d-4ada-8dbb-57fc06c2ffcc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:26.134603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50c280-f60d-4ada-8dbb-57fc06c2ffcc None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:26.137055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 444/882] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:26.142832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dbb2b-a002-4fbf-8771-2a77f1ae08f3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:26.143714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dbb2b-a002-4fbf-8771-2a77f1ae08f3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:26.144202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dbb2b-a002-4fbf-8771-2a77f1ae08f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:26.164207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-330dbb2b-a002-4fbf-8771-2a77f1ae08f3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:26.165866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-330dbb2b-a002-4fbf-8771-2a77f1ae08f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:26.195342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:26.254735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 439/883] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:26.273985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50c280-f60d-4ada-8dbb-57fc06c2ffcc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:26.348774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:26.388952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50c280-f60d-4ada-8dbb-57fc06c2ffcc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:26.427884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72473ac5-790b-4327-8149-96df3b5b4d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:26.428696 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72473ac5-790b-4327-8149-96df3b5b4d69 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:26.428979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72473ac5-790b-4327-8149-96df3b5b4d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:26.429256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72473ac5-790b-4327-8149-96df3b5b4d69 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:26.429942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72473ac5-790b-4327-8149-96df3b5b4d69 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:26.431448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72473ac5-790b-4327-8149-96df3b5b4d69 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:26.443065 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1f627-ab8b-4cd0-a2dd-e707b8ad26fc None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:26.443359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1f627-ab8b-4cd0-a2dd-e707b8ad26fc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:26.443636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1f627-ab8b-4cd0-a2dd-e707b8ad26fc None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-72473ac5-790b-4327-8149-96df3b5b4d69 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:26.444053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:26.444971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 16:07:26.445836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:26.447522 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 440/884] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:26.448877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 445/885] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:07:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:26.450611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4365b-357a-4027-a8c4-e9c932c4d4da None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:26.451127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4365b-357a-4027-a8c4-e9c932c4d4da None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:26.451474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72473ac5-790b-4327-8149-96df3b5b4d69 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:26.451834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4365b-357a-4027-a8c4-e9c932c4d4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:26.469406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-18d4365b-357a-4027-a8c4-e9c932c4d4da None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:26.471084 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18d4365b-357a-4027-a8c4-e9c932c4d4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:26.553737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 441/886] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:26.571384 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4365b-357a-4027-a8c4-e9c932c4d4da None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:26.581143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72473ac5-790b-4327-8149-96df3b5b4d69 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:26.613500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:26.620633 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ede3f-58c5-4ce2-96a2-1bfd2b4ddffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:26.621302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ede3f-58c5-4ce2-96a2-1bfd2b4ddffc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:26.621588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ede3f-58c5-4ce2-96a2-1bfd2b4ddffc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:26.621890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ede3f-58c5-4ce2-96a2-1bfd2b4ddffc None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:26.622609 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ede3f-58c5-4ce2-96a2-1bfd2b4ddffc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:26.622919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ede3f-58c5-4ce2-96a2-1bfd2b4ddffc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:26.624333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ede3f-58c5-4ce2-96a2-1bfd2b4ddffc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:26.632726 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 446/887] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:07:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:26.639545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ede3f-58c5-4ce2-96a2-1bfd2b4ddffc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:26.697425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:26.716640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4365b-357a-4027-a8c4-e9c932c4d4da None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:26.757128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ede3f-58c5-4ce2-96a2-1bfd2b4ddffc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:26.764864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a1543-923e-4b80-9c67-fb433463c6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:26.766626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a1543-923e-4b80-9c67-fb433463c6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:26.767090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a1543-923e-4b80-9c67-fb433463c6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:26.767573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a1543-923e-4b80-9c67-fb433463c6e8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:26.782749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a1543-923e-4b80-9c67-fb433463c6e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-115a1543-923e-4b80-9c67-fb433463c6e8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:26.791414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:26.793119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:26.794130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:26.797938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 442/888] 10.222.0.42 () {56 vars in 1120 bytes} [Wed May 25 16:07:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:26.799293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a1543-923e-4b80-9c67-fb433463c6e8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:26.805148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ded62-f86b-4066-a975-818340a92286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:26.806167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ded62-f86b-4066-a975-818340a92286 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:26.806663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ded62-f86b-4066-a975-818340a92286 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:26.807179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ded62-f86b-4066-a975-818340a92286 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:26.808474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ded62-f86b-4066-a975-818340a92286 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:26.809119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ded62-f86b-4066-a975-818340a92286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:26.811292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ded62-f86b-4066-a975-818340a92286 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:26.819110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 447/889] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:07:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:26.826741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ded62-f86b-4066-a975-818340a92286 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:26.894092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:26.915515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a1543-923e-4b80-9c67-fb433463c6e8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:26.945020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ded62-f86b-4066-a975-818340a92286 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:26.953108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8738029-063b-4c8c-971b-20f70e0ec23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:26.953823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8738029-063b-4c8c-971b-20f70e0ec23a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:26.954115 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8738029-063b-4c8c-971b-20f70e0ec23a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:26.954397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8738029-063b-4c8c-971b-20f70e0ec23a None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:26.955158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8738029-063b-4c8c-971b-20f70e0ec23a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:26.955470 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8738029-063b-4c8c-971b-20f70e0ec23a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:26.957443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8738029-063b-4c8c-971b-20f70e0ec23a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:26.969969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 443/890] 10.222.0.42 () {56 vars in 1115 bytes} [Wed May 25 16:07:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:26.971429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dccfe-489f-4322-8fec-10f8379a7bc3 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:26.971727 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dccfe-489f-4322-8fec-10f8379a7bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:26.972006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dccfe-489f-4322-8fec-10f8379a7bc3 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:26.973463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 444/891] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:26.975282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14dccfe-489f-4322-8fec-10f8379a7bc3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:26.992361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:26.998727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11817ec6-4d1e-4b6e-ae68-a2e66f52de19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:27.001407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11817ec6-4d1e-4b6e-ae68-a2e66f52de19 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.001407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11817ec6-4d1e-4b6e-ae68-a2e66f52de19 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.001407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11817ec6-4d1e-4b6e-ae68-a2e66f52de19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.001407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11817ec6-4d1e-4b6e-ae68-a2e66f52de19 None admin] RBAC: Adding query filter params (interface=public, service_id=2ee3e95e1db8417686b944e009afd5bb, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:27.001751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11817ec6-4d1e-4b6e-ae68-a2e66f52de19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:27.005801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11817ec6-4d1e-4b6e-ae68-a2e66f52de19 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:27.011426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 448/892] 10.222.0.42 () {56 vars in 1251 bytes} [Wed May 25 16:07:26 2022] GET /identity/v3/endpoints?service_id=2ee3e95e1db8417686b944e009afd5bb&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:27.012702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89b358-39df-4545-bbab-47fdbae16385 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.013139 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89b358-39df-4545-bbab-47fdbae16385 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.013425 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89b358-39df-4545-bbab-47fdbae16385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.034092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5b89b358-39df-4545-bbab-47fdbae16385 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:27.035606 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5b89b358-39df-4545-bbab-47fdbae16385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:27.098796 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14dccfe-489f-4322-8fec-10f8379a7bc3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:27.115968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 449/893] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:27.117034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89b358-39df-4545-bbab-47fdbae16385 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:27.145580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539cf57-730d-49f0-9992-2a318fab9350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:27.146453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539cf57-730d-49f0-9992-2a318fab9350 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.146739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539cf57-730d-49f0-9992-2a318fab9350 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.147023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539cf57-730d-49f0-9992-2a318fab9350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.160993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539cf57-730d-49f0-9992-2a318fab9350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:27.162583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539cf57-730d-49f0-9992-2a318fab9350 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:27.176254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e539cf57-730d-49f0-9992-2a318fab9350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:27.183967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:27.184899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:27.186387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:27.186387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:27.186387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:27.186387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:27.186387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 445/894] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:27.186666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539cf57-730d-49f0-9992-2a318fab9350 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:27.231978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89b358-39df-4545-bbab-47fdbae16385 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:27.256740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:27.268178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e707fa-42fe-4827-8288-46c0d75ff682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:27.268847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e707fa-42fe-4827-8288-46c0d75ff682 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.269129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e707fa-42fe-4827-8288-46c0d75ff682 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.269404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e707fa-42fe-4827-8288-46c0d75ff682 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.270119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e707fa-42fe-4827-8288-46c0d75ff682 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:27.271505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e707fa-42fe-4827-8288-46c0d75ff682 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:27.278545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 450/895] 10.222.0.42 () {56 vars in 1087 bytes} [Wed May 25 16:07:27 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:27.279812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e707fa-42fe-4827-8288-46c0d75ff682 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:27.313485 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539cf57-730d-49f0-9992-2a318fab9350 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:27.366098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9767e-9d00-4acc-994a-ab6ae58c3c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:27.366789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9767e-9d00-4acc-994a-ab6ae58c3c35 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.367075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9767e-9d00-4acc-994a-ab6ae58c3c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.367349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9767e-9d00-4acc-994a-ab6ae58c3c35 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.368087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9767e-9d00-4acc-994a-ab6ae58c3c35 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:27.369488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9767e-9d00-4acc-994a-ab6ae58c3c35 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:27.386558 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 446/896] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:07:27 2022] POST /identity/v3/services => generated 241 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:27.388334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a74a-9dc6-45d2-9494-25bd22bc49d6 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.388699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a74a-9dc6-45d2-9494-25bd22bc49d6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.389093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a74a-9dc6-45d2-9494-25bd22bc49d6 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.390932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 447/897] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:27.396949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca958b-bead-46fd-be8a-b0afb7f3a92c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.397310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca958b-bead-46fd-be8a-b0afb7f3a92c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.397500 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca958b-bead-46fd-be8a-b0afb7f3a92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.417588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c5ca958b-bead-46fd-be8a-b0afb7f3a92c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:27.418892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c5ca958b-bead-46fd-be8a-b0afb7f3a92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:27.424007 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e707fa-42fe-4827-8288-46c0d75ff682 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:27.467634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:27.500078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4c9bd-8ad9-4b21-95f0-55b61eb3aeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:27.500989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4c9bd-8ad9-4b21-95f0-55b61eb3aeda None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.501887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4c9bd-8ad9-4b21-95f0-55b61eb3aeda None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.501887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4c9bd-8ad9-4b21-95f0-55b61eb3aeda None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.502469 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4c9bd-8ad9-4b21-95f0-55b61eb3aeda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:27.502798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4c9bd-8ad9-4b21-95f0-55b61eb3aeda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:27.504210 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4c9bd-8ad9-4b21-95f0-55b61eb3aeda None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:27.517079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 448/898] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:27.522709 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 451/899] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:27.528543 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca958b-bead-46fd-be8a-b0afb7f3a92c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:27.550900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4c9bd-8ad9-4b21-95f0-55b61eb3aeda None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:27.647935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:27.718723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca958b-bead-46fd-be8a-b0afb7f3a92c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:27.724224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4c9bd-8ad9-4b21-95f0-55b61eb3aeda None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:27.761029 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8ced4-cee7-402c-8334-31a4c9faef71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:27.762096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8ced4-cee7-402c-8334-31a4c9faef71 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.762400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8ced4-cee7-402c-8334-31a4c9faef71 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.762685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8ced4-cee7-402c-8334-31a4c9faef71 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.768167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919d19f-6ca2-4ad9-89ea-5393f6680a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:27.769259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919d19f-6ca2-4ad9-89ea-5393f6680a3d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.769794 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919d19f-6ca2-4ad9-89ea-5393f6680a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.770329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919d19f-6ca2-4ad9-89ea-5393f6680a3d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.771025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8ced4-cee7-402c-8334-31a4c9faef71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:27.771422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919d19f-6ca2-4ad9-89ea-5393f6680a3d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:27.772477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8ced4-cee7-402c-8334-31a4c9faef71 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:27.773900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919d19f-6ca2-4ad9-89ea-5393f6680a3d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0919d19f-6ca2-4ad9-89ea-5393f6680a3d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:27.788556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:27.789646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 16:07:27.790501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e5f8ced4-cee7-402c-8334-31a4c9faef71 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:27.791320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:27.792863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:27.793824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:27.793824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:27.793824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:27.793824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:27.793824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:27.793824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:27.793824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:27.793824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:27.793824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:27.793824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:27.796926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 449/900] 10.222.0.42 () {56 vars in 1117 bytes} [Wed May 25 16:07:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:27.801824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8ced4-cee7-402c-8334-31a4c9faef71 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:27.802218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 452/901] 10.222.0.42 () {56 vars in 1132 bytes} [Wed May 25 16:07:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:27.806429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919d19f-6ca2-4ad9-89ea-5393f6680a3d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:27.858437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:27.927919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8ced4-cee7-402c-8334-31a4c9faef71 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:27.930650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919d19f-6ca2-4ad9-89ea-5393f6680a3d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:27.947114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:27.972594 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cfbdd-6d0d-4e09-a81f-11a173bbf9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:27.973926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cfbdd-6d0d-4e09-a81f-11a173bbf9be None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.974396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cfbdd-6d0d-4e09-a81f-11a173bbf9be None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.974648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6bbb6-7a80-4c6f-b220-b3e9fa666a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:27.975031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cfbdd-6d0d-4e09-a81f-11a173bbf9be None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.975550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6bbb6-7a80-4c6f-b220-b3e9fa666a5d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:27.975938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6bbb6-7a80-4c6f-b220-b3e9fa666a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:27.976230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cfbdd-6d0d-4e09-a81f-11a173bbf9be None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:27.976393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6bbb6-7a80-4c6f-b220-b3e9fa666a5d None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:27.976833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cfbdd-6d0d-4e09-a81f-11a173bbf9be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:27.977432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6bbb6-7a80-4c6f-b220-b3e9fa666a5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:27.977886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6bbb6-7a80-4c6f-b220-b3e9fa666a5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:27.978713 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cfbdd-6d0d-4e09-a81f-11a173bbf9be None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:27.980147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6bbb6-7a80-4c6f-b220-b3e9fa666a5d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:27.991141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 453/902] 10.222.0.42 () {56 vars in 1126 bytes} [Wed May 25 16:07:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:27.992423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cfbdd-6d0d-4e09-a81f-11a173bbf9be None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:28.036979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 450/903] 10.222.0.42 () {56 vars in 1149 bytes} [Wed May 25 16:07:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:28.041890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6bbb6-7a80-4c6f-b220-b3e9fa666a5d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:28.164369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cfbdd-6d0d-4e09-a81f-11a173bbf9be None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:28.168456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:28.209565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6bbb6-7a80-4c6f-b220-b3e9fa666a5d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:28.223491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c495b-f478-445f-8a74-eee378807c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:28.224158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c495b-f478-445f-8a74-eee378807c3c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:28.224443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c495b-f478-445f-8a74-eee378807c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:28.224721 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c495b-f478-445f-8a74-eee378807c3c None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:28.225437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c495b-f478-445f-8a74-eee378807c3c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:28.225889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c495b-f478-445f-8a74-eee378807c3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:28.227219 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c495b-f478-445f-8a74-eee378807c3c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:28.233774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 454/904] 10.222.0.42 () {56 vars in 1126 bytes} [Wed May 25 16:07:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:28.239334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c495b-f478-445f-8a74-eee378807c3c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:28.264811 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2122c59-3b00-4a64-b18d-b9f00a9a1292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:28.265981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2122c59-3b00-4a64-b18d-b9f00a9a1292 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:28.266556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2122c59-3b00-4a64-b18d-b9f00a9a1292 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:28.267031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2122c59-3b00-4a64-b18d-b9f00a9a1292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:28.312323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:28.347268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c495b-f478-445f-8a74-eee378807c3c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:28.347800 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2122c59-3b00-4a64-b18d-b9f00a9a1292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:28.349653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2122c59-3b00-4a64-b18d-b9f00a9a1292 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:28.383202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ecd3c-468b-43c6-bc30-46126a3e1be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:28.383864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ecd3c-468b-43c6-bc30-46126a3e1be1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:28.384207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ecd3c-468b-43c6-bc30-46126a3e1be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:28.384480 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ecd3c-468b-43c6-bc30-46126a3e1be1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:28.385319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ecd3c-468b-43c6-bc30-46126a3e1be1 None admin] RBAC: Adding query filter params (interface=public, service_id=901ed2d9680e4bc8ab8203ea36cfaf7e, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:28.386243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ecd3c-468b-43c6-bc30-46126a3e1be1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:28.388064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ecd3c-468b-43c6-bc30-46126a3e1be1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:28.398077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 455/905] 10.222.0.42 () {56 vars in 1251 bytes} [Wed May 25 16:07:28 2022] GET /identity/v3/endpoints?service_id=901ed2d9680e4bc8ab8203ea36cfaf7e&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:28.400105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ecd3c-468b-43c6-bc30-46126a3e1be1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:28.441118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:28.483319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 451/906] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:28 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 16:07:28.484469 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2122c59-3b00-4a64-b18d-b9f00a9a1292 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:28.543020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ecd3c-468b-43c6-bc30-46126a3e1be1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:28.563940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:28.595428 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475ee69-d6df-4019-9bdd-38b9ae32c0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:28.596709 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475ee69-d6df-4019-9bdd-38b9ae32c0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:28.597194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475ee69-d6df-4019-9bdd-38b9ae32c0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:28.597692 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475ee69-d6df-4019-9bdd-38b9ae32c0c6 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:28.598599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475ee69-d6df-4019-9bdd-38b9ae32c0c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:28.600294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475ee69-d6df-4019-9bdd-38b9ae32c0c6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:28.618220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 456/907] 10.222.0.42 () {56 vars in 1087 bytes} [Wed May 25 16:07:28 2022] GET /identity/v3/services => generated 1497 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:28.623158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2122c59-3b00-4a64-b18d-b9f00a9a1292 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:28.663589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7425a4-4391-4c88-a15d-e078d7ab26df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:28.664406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7425a4-4391-4c88-a15d-e078d7ab26df None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:28.664692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7425a4-4391-4c88-a15d-e078d7ab26df None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:28.664970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7425a4-4391-4c88-a15d-e078d7ab26df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:28.676886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7425a4-4391-4c88-a15d-e078d7ab26df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:28.678386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7425a4-4391-4c88-a15d-e078d7ab26df None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3b7425a4-4391-4c88-a15d-e078d7ab26df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:28.698576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:28.699546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:28.700432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:28.700432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:28.700432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:28.700432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:28.700432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 452/908] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:28.706877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475ee69-d6df-4019-9bdd-38b9ae32c0c6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:28.818277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:28.826731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475ee69-d6df-4019-9bdd-38b9ae32c0c6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:28.867490 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a300320-c5f1-495c-88ff-99fd77532adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:28.868206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a300320-c5f1-495c-88ff-99fd77532adf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:28.868493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a300320-c5f1-495c-88ff-99fd77532adf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:28.868768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a300320-c5f1-495c-88ff-99fd77532adf None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:28.869495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a300320-c5f1-495c-88ff-99fd77532adf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:28.869835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a300320-c5f1-495c-88ff-99fd77532adf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:28.871438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a300320-c5f1-495c-88ff-99fd77532adf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:28.883854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 457/909] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:28.890966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7425a4-4391-4c88-a15d-e078d7ab26df None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:29.008105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7425a4-4391-4c88-a15d-e078d7ab26df None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:29.033021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:29.046009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdf387-3806-4774-8655-58e32f9d98a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:29.046816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdf387-3806-4774-8655-58e32f9d98a7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:29.047101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdf387-3806-4774-8655-58e32f9d98a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:29.047379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdf387-3806-4774-8655-58e32f9d98a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:29.054616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefdf387-3806-4774-8655-58e32f9d98a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:29.056069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefdf387-3806-4774-8655-58e32f9d98a7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-fefdf387-3806-4774-8655-58e32f9d98a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:29.071731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:29.072665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:29.073970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:29.073970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:29.073970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:29.073970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:29.073970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:29.073970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:29.073970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:29.073970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:29.073970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 453/910] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:29.080806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a300320-c5f1-495c-88ff-99fd77532adf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:29.188581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a300320-c5f1-495c-88ff-99fd77532adf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:29.224221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9ed8d-1e8e-4f18-bea1-c6a956de7d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:29.224974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9ed8d-1e8e-4f18-bea1-c6a956de7d87 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:29.225167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9ed8d-1e8e-4f18-bea1-c6a956de7d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:29.225445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9ed8d-1e8e-4f18-bea1-c6a956de7d87 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:29.226261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9ed8d-1e8e-4f18-bea1-c6a956de7d87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:29.226573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9ed8d-1e8e-4f18-bea1-c6a956de7d87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:29.228007 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9ed8d-1e8e-4f18-bea1-c6a956de7d87 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:29.241494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 458/911] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:29.247786 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdf387-3806-4774-8655-58e32f9d98a7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:29.359794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdf387-3806-4774-8655-58e32f9d98a7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:29.402010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a1c67-bb46-4122-88f5-88c4d58bc3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:29.402683 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a1c67-bb46-4122-88f5-88c4d58bc3ea None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:29.402967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a1c67-bb46-4122-88f5-88c4d58bc3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:29.403240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a1c67-bb46-4122-88f5-88c4d58bc3ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:29.404067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a1c67-bb46-4122-88f5-88c4d58bc3ea None admin] RBAC: Adding query filter params (role.id=73f22629793d47f086feb1e90de17023, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=730cb89f7a214fde9c5040674323f8db) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:29.404379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a1c67-bb46-4122-88f5-88c4d58bc3ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:29.405837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a1c67-bb46-4122-88f5-88c4d58bc3ea None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:29.412713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 454/912] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:29 2022] GET /identity/v3/role_assignments?user.id=730cb89f7a214fde9c5040674323f8db&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=73f22629793d47f086feb1e90de17023 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:29.654749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f05536-8eda-4238-99e8-3a85a452b095 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:29.655091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f05536-8eda-4238-99e8-3a85a452b095 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:29.655266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f05536-8eda-4238-99e8-3a85a452b095 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:29.656893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 459/913] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:29.662580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e71173-6439-4a27-9428-769d24e84678 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:29.662871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e71173-6439-4a27-9428-769d24e84678 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:29.663151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e71173-6439-4a27-9428-769d24e84678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:29.681334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-12e71173-6439-4a27-9428-769d24e84678 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:29.682655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-12e71173-6439-4a27-9428-769d24e84678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:29.734427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:29.762820 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 455/914] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:29.785689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f05536-8eda-4238-99e8-3a85a452b095 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:29.802314 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:29.882769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be518f2-13d1-4bba-a313-7931b2191332 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:29.883057 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be518f2-13d1-4bba-a313-7931b2191332 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:29.883334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be518f2-13d1-4bba-a313-7931b2191332 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:29.884978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 456/915] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:29.890951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8300df-1a6b-4c58-bd04-dcc6620f3391 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:29.891243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8300df-1a6b-4c58-bd04-dcc6620f3391 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:29.891524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8300df-1a6b-4c58-bd04-dcc6620f3391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:29.898035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f05536-8eda-4238-99e8-3a85a452b095 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:29.909541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4a8300df-1a6b-4c58-bd04-dcc6620f3391 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:29.911048 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4a8300df-1a6b-4c58-bd04-dcc6620f3391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:29.940396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d3626-5135-4187-9e08-12f780b5edbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:29.941232 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d3626-5135-4187-9e08-12f780b5edbf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:29.941519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d3626-5135-4187-9e08-12f780b5edbf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:29.941846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d3626-5135-4187-9e08-12f780b5edbf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:29.942533 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d3626-5135-4187-9e08-12f780b5edbf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:29.943941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d3626-5135-4187-9e08-12f780b5edbf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-485d3626-5135-4187-9e08-12f780b5edbf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:29.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:29.958984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:29.959825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 460/916] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:07:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:29.966816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d3626-5135-4187-9e08-12f780b5edbf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:30.003931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 457/917] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:30.026865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8300df-1a6b-4c58-bd04-dcc6620f3391 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:30.056157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:30.083096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d3626-5135-4187-9e08-12f780b5edbf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:30.122960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeb7b7-74d4-4ad0-ad43-16d822acc477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:30.123750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeb7b7-74d4-4ad0-ad43-16d822acc477 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:30.124110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeb7b7-74d4-4ad0-ad43-16d822acc477 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:30.124458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeb7b7-74d4-4ad0-ad43-16d822acc477 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:30.125238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adeb7b7-74d4-4ad0-ad43-16d822acc477 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:30.125623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adeb7b7-74d4-4ad0-ad43-16d822acc477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:30.127191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adeb7b7-74d4-4ad0-ad43-16d822acc477 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:30.137355 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 461/918] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:07:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:30.138665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeb7b7-74d4-4ad0-ad43-16d822acc477 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:30.149217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8300df-1a6b-4c58-bd04-dcc6620f3391 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:30.161463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:30.204026 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f4676-459d-4b96-8b8d-d7f2d9ae315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:30.205423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f4676-459d-4b96-8b8d-d7f2d9ae315a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:30.206274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f4676-459d-4b96-8b8d-d7f2d9ae315a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:30.207017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f4676-459d-4b96-8b8d-d7f2d9ae315a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:30.207870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f4676-459d-4b96-8b8d-d7f2d9ae315a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:30.211295 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f4676-459d-4b96-8b8d-d7f2d9ae315a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-738f4676-459d-4b96-8b8d-d7f2d9ae315a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:30.240114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:30.241669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:30.243391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:30.244645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 458/919] 10.222.0.42 () {56 vars in 1126 bytes} [Wed May 25 16:07:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:30.252429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f4676-459d-4b96-8b8d-d7f2d9ae315a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:30.275806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeb7b7-74d4-4ad0-ad43-16d822acc477 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:30.313253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9672a-1334-4032-a3af-8d346f04f5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:30.313789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9672a-1334-4032-a3af-8d346f04f5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:30.314078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9672a-1334-4032-a3af-8d346f04f5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:30.314359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9672a-1334-4032-a3af-8d346f04f5a2 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:30.315070 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9672a-1334-4032-a3af-8d346f04f5a2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:30.315377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9672a-1334-4032-a3af-8d346f04f5a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:30.316752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9672a-1334-4032-a3af-8d346f04f5a2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:30.325858 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 462/920] 10.222.0.42 () {56 vars in 1114 bytes} [Wed May 25 16:07:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:30.327977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9672a-1334-4032-a3af-8d346f04f5a2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:30.337518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:30.391443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f4676-459d-4b96-8b8d-d7f2d9ae315a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:30.446803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c08f2a-d4de-41bd-bd43-eb8e22f8cbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:30.447567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c08f2a-d4de-41bd-bd43-eb8e22f8cbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:30.447921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c08f2a-d4de-41bd-bd43-eb8e22f8cbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:30.448956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c08f2a-d4de-41bd-bd43-eb8e22f8cbc4 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:30.449253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9672a-1334-4032-a3af-8d346f04f5a2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:30.450020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c08f2a-d4de-41bd-bd43-eb8e22f8cbc4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:30.450402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c08f2a-d4de-41bd-bd43-eb8e22f8cbc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:30.451854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c08f2a-d4de-41bd-bd43-eb8e22f8cbc4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:30.457776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 459/921] 10.222.0.42 () {56 vars in 1122 bytes} [Wed May 25 16:07:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:30.463261 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c08f2a-d4de-41bd-bd43-eb8e22f8cbc4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:30.504051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7d402-893e-40dc-9bd0-a2e2000049c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:30.504820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7d402-893e-40dc-9bd0-a2e2000049c4 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:30.505112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7d402-893e-40dc-9bd0-a2e2000049c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:30.505392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7d402-893e-40dc-9bd0-a2e2000049c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:30.506190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7d402-893e-40dc-9bd0-a2e2000049c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:30.507581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7d402-893e-40dc-9bd0-a2e2000049c4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:30.507874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:30.538026 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 463/922] 10.222.0.42 () {60 vars in 1144 bytes} [Wed May 25 16:07:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:30.591533 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c08f2a-d4de-41bd-bd43-eb8e22f8cbc4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:30.638510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e6670-8dd1-4e12-b975-233334a0f0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:30.639604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6670-8dd1-4e12-b975-233334a0f0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:30.640100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6670-8dd1-4e12-b975-233334a0f0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:30.640584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6670-8dd1-4e12-b975-233334a0f0e5 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:30.641643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e6670-8dd1-4e12-b975-233334a0f0e5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:30.642209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e6670-8dd1-4e12-b975-233334a0f0e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:30.643851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e6670-8dd1-4e12-b975-233334a0f0e5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:30.650970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 460/923] 10.222.0.42 () {56 vars in 1122 bytes} [Wed May 25 16:07:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:30.658090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7d402-893e-40dc-9bd0-a2e2000049c4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:30.731309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:30.774714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7d402-893e-40dc-9bd0-a2e2000049c4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:30.813819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7745ee-ce09-48a7-baf2-70bf0616df79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:30.814496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7745ee-ce09-48a7-baf2-70bf0616df79 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:30.814779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7745ee-ce09-48a7-baf2-70bf0616df79 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:30.815056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7745ee-ce09-48a7-baf2-70bf0616df79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:30.815865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7745ee-ce09-48a7-baf2-70bf0616df79 None admin] RBAC: Adding query filter params (interface=public, service_id=9cabd0daf45a401caca2a9f1da7b19fe, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:30.817164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7745ee-ce09-48a7-baf2-70bf0616df79 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:30.818586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7745ee-ce09-48a7-baf2-70bf0616df79 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:30.825042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 464/924] 10.222.0.42 () {56 vars in 1251 bytes} [Wed May 25 16:07:30 2022] GET /identity/v3/endpoints?service_id=9cabd0daf45a401caca2a9f1da7b19fe&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:30.830577 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e6670-8dd1-4e12-b975-233334a0f0e5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:30.938031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:30.945647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e6670-8dd1-4e12-b975-233334a0f0e5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:30.983123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3099a5-088a-4d8c-8530-41dd1a3c371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:30.984192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3099a5-088a-4d8c-8530-41dd1a3c371f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:30.984478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3099a5-088a-4d8c-8530-41dd1a3c371f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:30.984757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3099a5-088a-4d8c-8530-41dd1a3c371f None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:30.985435 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3099a5-088a-4d8c-8530-41dd1a3c371f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:30.986845 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3099a5-088a-4d8c-8530-41dd1a3c371f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:30.993663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 461/925] 10.222.0.42 () {56 vars in 1087 bytes} [Wed May 25 16:07:30 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:31.117231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031c06d-791e-445d-93ef-ceae6afed394 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:31.117589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031c06d-791e-445d-93ef-ceae6afed394 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:31.117828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031c06d-791e-445d-93ef-ceae6afed394 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:31.119550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 465/926] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:31.125751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a35fdc-7c0a-41d2-aac1-927765694dc0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:31.126063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a35fdc-7c0a-41d2-aac1-927765694dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:31.126341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a35fdc-7c0a-41d2-aac1-927765694dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:31.145432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a1a35fdc-7c0a-41d2-aac1-927765694dc0 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:31.146731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a1a35fdc-7c0a-41d2-aac1-927765694dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:31.222789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 462/927] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:31.247407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031c06d-791e-445d-93ef-ceae6afed394 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:31.361344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031c06d-791e-445d-93ef-ceae6afed394 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:31.389679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:31.399291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6500aa-876c-4989-ad9f-c350481ddc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:31.400116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6500aa-876c-4989-ad9f-c350481ddc9d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:31.400402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6500aa-876c-4989-ad9f-c350481ddc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:31.400674 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6500aa-876c-4989-ad9f-c350481ddc9d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:31.401913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6500aa-876c-4989-ad9f-c350481ddc9d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:31.403521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6500aa-876c-4989-ad9f-c350481ddc9d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4b6500aa-876c-4989-ad9f-c350481ddc9d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:31.416884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:31.418151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 16:07:31.419001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:31.423339 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 466/928] 10.222.0.42 () {56 vars in 1132 bytes} [Wed May 25 16:07:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:31.424715 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a35fdc-7c0a-41d2-aac1-927765694dc0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:31.525290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:31.541501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a35fdc-7c0a-41d2-aac1-927765694dc0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:31.580033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6168a-edaf-4309-b289-bce3ea66731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:31.580721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6168a-edaf-4309-b289-bce3ea66731f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:31.581005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6168a-edaf-4309-b289-bce3ea66731f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:31.581282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6168a-edaf-4309-b289-bce3ea66731f None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:31.582021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6168a-edaf-4309-b289-bce3ea66731f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:31.582331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6168a-edaf-4309-b289-bce3ea66731f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:31.583699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6168a-edaf-4309-b289-bce3ea66731f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:31.589945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 463/929] 10.222.0.42 () {56 vars in 1126 bytes} [Wed May 25 16:07:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:31.595605 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6500aa-876c-4989-ad9f-c350481ddc9d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:31.701619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6500aa-876c-4989-ad9f-c350481ddc9d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:31.737696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3740188-370d-451a-a61b-0f00dc75aa4a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:31.738038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3740188-370d-451a-a61b-0f00dc75aa4a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:31.738343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3740188-370d-451a-a61b-0f00dc75aa4a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:31.739573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7361ad-cbb0-4508-807c-204b6646d9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:31.740011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 464/930] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:07:31.740248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7361ad-cbb0-4508-807c-204b6646d9fd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:31.740534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7361ad-cbb0-4508-807c-204b6646d9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:31.740812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7361ad-cbb0-4508-807c-204b6646d9fd None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:31.741530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7361ad-cbb0-4508-807c-204b6646d9fd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:31.741869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7361ad-cbb0-4508-807c-204b6646d9fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:31.743253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7361ad-cbb0-4508-807c-204b6646d9fd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:31.745084 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41aa6bb-0cbe-4273-97cc-fd0b31444fda None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:31.745480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41aa6bb-0cbe-4273-97cc-fd0b31444fda None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:31.746028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41aa6bb-0cbe-4273-97cc-fd0b31444fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:31.749202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 467/931] 10.222.0.42 () {56 vars in 1126 bytes} [Wed May 25 16:07:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:31.755433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7361ad-cbb0-4508-807c-204b6646d9fd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:31.769441 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e41aa6bb-0cbe-4273-97cc-fd0b31444fda None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:31.771014 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e41aa6bb-0cbe-4273-97cc-fd0b31444fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:31.818513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:31.873737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 465/932] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:31.876984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7361ad-cbb0-4508-807c-204b6646d9fd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:31.891393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41aa6bb-0cbe-4273-97cc-fd0b31444fda None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:31.919852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9432-1b21-42a4-9d22-b119da93aee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:31.920898 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9432-1b21-42a4-9d22-b119da93aee9 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:31.921380 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9432-1b21-42a4-9d22-b119da93aee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:31.921909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9432-1b21-42a4-9d22-b119da93aee9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:31.923173 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a9432-1b21-42a4-9d22-b119da93aee9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:31.924817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a9432-1b21-42a4-9d22-b119da93aee9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:31.954213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 468/933] 10.222.0.42 () {60 vars in 1144 bytes} [Wed May 25 16:07:31 2022] POST /identity/v3/endpoints => generated 347 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:31.989265 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:32.009836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41aa6bb-0cbe-4273-97cc-fd0b31444fda None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:32.056137 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a525be7-8fd8-4149-8ea2-b32f5289729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:32.056989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a525be7-8fd8-4149-8ea2-b32f5289729b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:32.057257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a525be7-8fd8-4149-8ea2-b32f5289729b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:32.057599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a525be7-8fd8-4149-8ea2-b32f5289729b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:32.070248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a525be7-8fd8-4149-8ea2-b32f5289729b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0a525be7-8fd8-4149-8ea2-b32f5289729b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:32.079112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:32.080141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:32.081006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 466/934] 10.222.0.42 () {56 vars in 1120 bytes} [Wed May 25 16:07:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:32.088356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9432-1b21-42a4-9d22-b119da93aee9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:32.207147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9432-1b21-42a4-9d22-b119da93aee9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:32.219906 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:32.249400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50837547-fb1b-4a97-8d72-8ff4c88a798d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:32.250468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50837547-fb1b-4a97-8d72-8ff4c88a798d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:32.250965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50837547-fb1b-4a97-8d72-8ff4c88a798d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:32.251445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50837547-fb1b-4a97-8d72-8ff4c88a798d None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:32.252387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50837547-fb1b-4a97-8d72-8ff4c88a798d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:32.252898 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50837547-fb1b-4a97-8d72-8ff4c88a798d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:32.254551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50837547-fb1b-4a97-8d72-8ff4c88a798d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:32.264964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 469/935] 10.222.0.42 () {56 vars in 1115 bytes} [Wed May 25 16:07:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:32.270943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a525be7-8fd8-4149-8ea2-b32f5289729b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:32.379784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a525be7-8fd8-4149-8ea2-b32f5289729b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:32.417090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32723013-70f6-498d-ba1a-848821c210ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:32.418608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723013-70f6-498d-ba1a-848821c210ef None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:32.419113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723013-70f6-498d-ba1a-848821c210ef None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:32.419648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723013-70f6-498d-ba1a-848821c210ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:32.427560 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:32.434640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32723013-70f6-498d-ba1a-848821c210ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:32.436423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32723013-70f6-498d-ba1a-848821c210ef None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-32723013-70f6-498d-ba1a-848821c210ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:32.454652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:32.455576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:32.456948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:32.456948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:32.456948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:32.456948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:32.456948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 467/936] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:32.461821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50837547-fb1b-4a97-8d72-8ff4c88a798d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:32.572479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50837547-fb1b-4a97-8d72-8ff4c88a798d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:32.608453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3662ea-733d-4d9c-9bb0-5b1a473f993a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:32.609113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3662ea-733d-4d9c-9bb0-5b1a473f993a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:32.609396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3662ea-733d-4d9c-9bb0-5b1a473f993a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:32.609665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3662ea-733d-4d9c-9bb0-5b1a473f993a None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:32.610413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3662ea-733d-4d9c-9bb0-5b1a473f993a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:32.610714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3662ea-733d-4d9c-9bb0-5b1a473f993a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:32.612084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3662ea-733d-4d9c-9bb0-5b1a473f993a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:32.625438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 470/937] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:32.630862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32723013-70f6-498d-ba1a-848821c210ef None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:32.740113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32723013-70f6-498d-ba1a-848821c210ef None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:32.780708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a223b4-af37-46c5-a1f9-2b9c772c7a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:32.781945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a223b4-af37-46c5-a1f9-2b9c772c7a2e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:32.782440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a223b4-af37-46c5-a1f9-2b9c772c7a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:32.782958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a223b4-af37-46c5-a1f9-2b9c772c7a2e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:32.794061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a223b4-af37-46c5-a1f9-2b9c772c7a2e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:32.795946 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a223b4-af37-46c5-a1f9-2b9c772c7a2e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e3a223b4-af37-46c5-a1f9-2b9c772c7a2e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:32.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:32.814062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:32.815000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:32.815000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:32.815000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:32.815000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:32.815000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:32.815000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:32.815000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:32.815000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:32.815000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:32.815000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:32.817938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 468/938] 10.222.0.42 () {56 vars in 1117 bytes} [Wed May 25 16:07:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:32.827170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3662ea-733d-4d9c-9bb0-5b1a473f993a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:32.858774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:32.941925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3662ea-733d-4d9c-9bb0-5b1a473f993a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:32.978712 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e89c66-229f-4650-b94a-798256699f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:32.980017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89c66-229f-4650-b94a-798256699f07 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:32.980519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89c66-229f-4650-b94a-798256699f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:32.980955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89c66-229f-4650-b94a-798256699f07 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:32.981834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e89c66-229f-4650-b94a-798256699f07 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:32.982144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e89c66-229f-4650-b94a-798256699f07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:32.983577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e89c66-229f-4650-b94a-798256699f07 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:33.001871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cefb2-2da6-4625-bf33-4dcd6bba03cf None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.002177 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cefb2-2da6-4625-bf33-4dcd6bba03cf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.002423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cefb2-2da6-4625-bf33-4dcd6bba03cf None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.003353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 471/939] 10.222.0.42 () {56 vars in 1149 bytes} [Wed May 25 16:07:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:33.003998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 469/940] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:33.009478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8cefb2-2da6-4625-bf33-4dcd6bba03cf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:33.009768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875d901-ff83-4a24-9896-cde45d8f629d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.010187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875d901-ff83-4a24-9896-cde45d8f629d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.010958 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875d901-ff83-4a24-9896-cde45d8f629d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.034506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3875d901-ff83-4a24-9896-cde45d8f629d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:33.036668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3875d901-ff83-4a24-9896-cde45d8f629d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:33.074398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:33.113315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 472/941] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:33.133663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8cefb2-2da6-4625-bf33-4dcd6bba03cf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:33.137567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875d901-ff83-4a24-9896-cde45d8f629d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:33.170915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:33.177244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7fe9f-2dd6-485c-b4f0-7022f4f34749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:33.178499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7fe9f-2dd6-485c-b4f0-7022f4f34749 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.179007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7fe9f-2dd6-485c-b4f0-7022f4f34749 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.179484 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7fe9f-2dd6-485c-b4f0-7022f4f34749 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.192894 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7fe9f-2dd6-485c-b4f0-7022f4f34749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:33.194845 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7fe9f-2dd6-485c-b4f0-7022f4f34749 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-09b7fe9f-2dd6-485c-b4f0-7022f4f34749 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:33.214935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:33.216022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:33.216931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:33.216931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:33.216931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:33.216931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:33.219487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 470/942] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:33.226964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7fe9f-2dd6-485c-b4f0-7022f4f34749 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:33.270002 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875d901-ff83-4a24-9896-cde45d8f629d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:33.311253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6937d1e-f460-4e35-b9b4-9d7a966b78c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:33.312431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6937d1e-f460-4e35-b9b4-9d7a966b78c7 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.313150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6937d1e-f460-4e35-b9b4-9d7a966b78c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.313986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6937d1e-f460-4e35-b9b4-9d7a966b78c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.315202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6937d1e-f460-4e35-b9b4-9d7a966b78c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:33.317669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6937d1e-f460-4e35-b9b4-9d7a966b78c7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:33.319314 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: WARNING keystone.server.flask.common [None req-c6937d1e-f460-4e35-b9b4-9d7a966b78c7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 25 16:07:33.340204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.sql.core [None req-c6937d1e-f460-4e35-b9b4-9d7a966b78c7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 16:07:33.340204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 16:07:33.340204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [parameters: {'id': '7d6017b7e134446790675222f957514c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 16:07:33.340204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80168) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c6937d1e-f460-4e35-b9b4-9d7a966b78c7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7d6017b7e134446790675222f957514c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7d6017b7e134446790675222f957514c. May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 16:07:33.360985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 25 16:07:33.362282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return project_ref.to_dict() May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application yield resource May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.rollback() May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.force_reraise() May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise self.value May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application session.commit() May 25 16:07:33.363575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._prepare_impl() May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.flush() May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._flush(objects) May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application compat.raise_( May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application flush_context.execute() May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 16:07:33.366268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rec.execute(self) May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 16:07:33.368354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 16:07:33.371849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [parameters: {'id': '7d6017b7e134446790675222f957514c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:33.375252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:33.378074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict( May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7d6017b7e134446790675222f957514c. May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 473/943] 10.222.0.42 () {60 vars in 1140 bytes} [Wed May 25 16:07:33 2022] POST /identity/v3/projects => generated 262 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb8121-1afd-4a39-b8cd-1bb7409455ca None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.380264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb8121-1afd-4a39-b8cd-1bb7409455ca None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.382924 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb8121-1afd-4a39-b8cd-1bb7409455ca None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.382924 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 474/944] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:33.382924 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb8121-1afd-4a39-b8cd-1bb7409455ca None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:33.383675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7fe9f-2dd6-485c-b4f0-7022f4f34749 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:33.383675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:33.419978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d9e64-e836-4a28-9068-717ef1d8edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:33.420661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d9e64-e836-4a28-9068-717ef1d8edb3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.420942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d9e64-e836-4a28-9068-717ef1d8edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.421219 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d9e64-e836-4a28-9068-717ef1d8edb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.422029 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d9e64-e836-4a28-9068-717ef1d8edb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:33.422336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d9e64-e836-4a28-9068-717ef1d8edb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:33.423954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d9e64-e836-4a28-9068-717ef1d8edb3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:33.446087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 471/945] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:33.447227 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa2305-09cf-4c34-b73e-265b67023a47 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.447517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa2305-09cf-4c34-b73e-265b67023a47 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.447861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa2305-09cf-4c34-b73e-265b67023a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.458004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:33.468232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e7aa2305-09cf-4c34-b73e-265b67023a47 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:33.469437 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e7aa2305-09cf-4c34-b73e-265b67023a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:33.487659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb8121-1afd-4a39-b8cd-1bb7409455ca None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:33.523160 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152b358-1c48-4ee4-933a-496e20d4c32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:33.524251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152b358-1c48-4ee4-933a-496e20d4c32d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.524536 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152b358-1c48-4ee4-933a-496e20d4c32d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.524811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152b358-1c48-4ee4-933a-496e20d4c32d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.532919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152b358-1c48-4ee4-933a-496e20d4c32d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:33.534849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152b358-1c48-4ee4-933a-496e20d4c32d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:33.541024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 472/946] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:33.541903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa2305-09cf-4c34-b73e-265b67023a47 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a152b358-1c48-4ee4-933a-496e20d4c32d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:33.562170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:33.563145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:33.564035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:33.564035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:33.564035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:33.564035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:33.564035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:33.564035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:33.564035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:33.564035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:33.566707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 475/947] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:33.567932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152b358-1c48-4ee4-933a-496e20d4c32d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:33.645666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:33.656472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa2305-09cf-4c34-b73e-265b67023a47 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:33.696405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b08904-4f7a-421f-80ed-905f103297ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:33.697129 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08904-4f7a-421f-80ed-905f103297ab None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.697414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08904-4f7a-421f-80ed-905f103297ab None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.697689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08904-4f7a-421f-80ed-905f103297ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.702307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152b358-1c48-4ee4-933a-496e20d4c32d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:33.704888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b08904-4f7a-421f-80ed-905f103297ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:33.706368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b08904-4f7a-421f-80ed-905f103297ab None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e7b08904-4f7a-421f-80ed-905f103297ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:33.720461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:33.721407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:33.722365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:33.722365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:33.722365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:33.722365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:33.722365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:33.722365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:33.722365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:33.722365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:33.722365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 473/948] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:33.722862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b08904-4f7a-421f-80ed-905f103297ab None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:33.750056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d231b12-d36b-49de-8c8d-c00f42b7adfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:33.750871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d231b12-d36b-49de-8c8d-c00f42b7adfa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.751155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d231b12-d36b-49de-8c8d-c00f42b7adfa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.751640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d231b12-d36b-49de-8c8d-c00f42b7adfa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.752079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d231b12-d36b-49de-8c8d-c00f42b7adfa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:33.753453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d231b12-d36b-49de-8c8d-c00f42b7adfa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:33.756155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9d231b12-d36b-49de-8c8d-c00f42b7adfa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:33.767113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:33.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:33.769198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:33.777836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 476/949] 10.222.0.42 () {56 vars in 1126 bytes} [Wed May 25 16:07:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:33.777836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d231b12-d36b-49de-8c8d-c00f42b7adfa None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:33.843267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b08904-4f7a-421f-80ed-905f103297ab None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:33.881301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b21f9f-ac5a-4909-94f6-59f3ea9afe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:33.882000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b21f9f-ac5a-4909-94f6-59f3ea9afe35 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.882283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b21f9f-ac5a-4909-94f6-59f3ea9afe35 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.882556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b21f9f-ac5a-4909-94f6-59f3ea9afe35 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.883312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b21f9f-ac5a-4909-94f6-59f3ea9afe35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:33.883614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b21f9f-ac5a-4909-94f6-59f3ea9afe35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:33.885044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b21f9f-ac5a-4909-94f6-59f3ea9afe35 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:33.888085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d231b12-d36b-49de-8c8d-c00f42b7adfa None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:33.905452 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 474/950] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:33 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:33.906677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b21f9f-ac5a-4909-94f6-59f3ea9afe35 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:33.925033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:33.958186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479cb84-131c-4296-8624-15edafea5d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:33.958774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479cb84-131c-4296-8624-15edafea5d26 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:33.959055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479cb84-131c-4296-8624-15edafea5d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:33.959329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479cb84-131c-4296-8624-15edafea5d26 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:33.960131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479cb84-131c-4296-8624-15edafea5d26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:33.960438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479cb84-131c-4296-8624-15edafea5d26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:33.961916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479cb84-131c-4296-8624-15edafea5d26 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:33.984901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 477/951] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:33.985666 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479cb84-131c-4296-8624-15edafea5d26 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:34.036747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b21f9f-ac5a-4909-94f6-59f3ea9afe35 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:34.060207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:34.078931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbcebd-e603-4c5a-99dd-aea7d4eec182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:34.080059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbcebd-e603-4c5a-99dd-aea7d4eec182 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:34.080558 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbcebd-e603-4c5a-99dd-aea7d4eec182 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:34.081072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbcebd-e603-4c5a-99dd-aea7d4eec182 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:34.082063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbcebd-e603-4c5a-99dd-aea7d4eec182 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:34.082619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbcebd-e603-4c5a-99dd-aea7d4eec182 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:34.084253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbcebd-e603-4c5a-99dd-aea7d4eec182 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:34.095916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 475/952] 10.222.0.42 () {56 vars in 1122 bytes} [Wed May 25 16:07:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:34.097291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbcebd-e603-4c5a-99dd-aea7d4eec182 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:34.111791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479cb84-131c-4296-8624-15edafea5d26 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:34.152950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0bb3b-43f1-4c8d-8acc-a2611d4407a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:34.153886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0bb3b-43f1-4c8d-8acc-a2611d4407a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:34.154168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0bb3b-43f1-4c8d-8acc-a2611d4407a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:34.154449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0bb3b-43f1-4c8d-8acc-a2611d4407a4 None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/730cb89f7a214fde9c5040674323f8db/roles/73f22629793d47f086feb1e90de17023` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:34.157479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0bb3b-43f1-4c8d-8acc-a2611d4407a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=730cb89f7a214fde9c5040674323f8db, role_id=73f22629793d47f086feb1e90de17023)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:34.160369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0bb3b-43f1-4c8d-8acc-a2611d4407a4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:34.174970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 478/953] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:07:33 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/730cb89f7a214fde9c5040674323f8db/roles/73f22629793d47f086feb1e90de17023 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:34.219495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:34.251093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbcebd-e603-4c5a-99dd-aea7d4eec182 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:34.302421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415db90f-4c7d-4275-b7e0-031eb7ad785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:34.303112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415db90f-4c7d-4275-b7e0-031eb7ad785f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:34.303395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415db90f-4c7d-4275-b7e0-031eb7ad785f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:34.303672 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415db90f-4c7d-4275-b7e0-031eb7ad785f None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:34.304399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415db90f-4c7d-4275-b7e0-031eb7ad785f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:34.304704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415db90f-4c7d-4275-b7e0-031eb7ad785f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:34.306165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415db90f-4c7d-4275-b7e0-031eb7ad785f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:34.312538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 476/954] 10.222.0.42 () {56 vars in 1122 bytes} [Wed May 25 16:07:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:34.318955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0bb3b-43f1-4c8d-8acc-a2611d4407a4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:34.441695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0bb3b-43f1-4c8d-8acc-a2611d4407a4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:34.467681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:34.480700 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34811de-d6d7-461f-954e-b5ec7a5ef405 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:34.481200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34811de-d6d7-461f-954e-b5ec7a5ef405 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:34.481280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34811de-d6d7-461f-954e-b5ec7a5ef405 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:34.482924 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 477/955] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:07:34.487331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea25b7a-1465-45d3-b85a-c8c4d0f9e7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:34.488329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea25b7a-1465-45d3-b85a-c8c4d0f9e7ab None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:34.489617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea25b7a-1465-45d3-b85a-c8c4d0f9e7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:34.491303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68f63f-261c-4895-b9c1-f849918fa85c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:34.491876 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea25b7a-1465-45d3-b85a-c8c4d0f9e7ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:34.492675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea25b7a-1465-45d3-b85a-c8c4d0f9e7ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:34.494111 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea25b7a-1465-45d3-b85a-c8c4d0f9e7ab None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:34.494677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68f63f-261c-4895-b9c1-f849918fa85c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:34.495874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68f63f-261c-4895-b9c1-f849918fa85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:34.518827 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-aa68f63f-261c-4895-b9c1-f849918fa85c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:34.520490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aa68f63f-261c-4895-b9c1-f849918fa85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:34.523458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 479/956] 10.222.0.42 () {60 vars in 1144 bytes} [Wed May 25 16:07:34 2022] POST /identity/v3/endpoints => generated 349 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:34.621837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 478/957] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:34.646635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea25b7a-1465-45d3-b85a-c8c4d0f9e7ab None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:34.804142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea25b7a-1465-45d3-b85a-c8c4d0f9e7ab None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:34.857530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b2279-9ab7-41c8-9815-ee698855adb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:34.858245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b2279-9ab7-41c8-9815-ee698855adb9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:34.858582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b2279-9ab7-41c8-9815-ee698855adb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:34.858866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b2279-9ab7-41c8-9815-ee698855adb9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:34.859532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b2279-9ab7-41c8-9815-ee698855adb9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:34.860918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b2279-9ab7-41c8-9815-ee698855adb9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-563b2279-9ab7-41c8-9815-ee698855adb9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:34.877087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:34.878417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 16:07:34.879306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:34.881888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 480/958] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:34.887486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68f63f-261c-4895-b9c1-f849918fa85c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:34.974732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:35.012556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68f63f-261c-4895-b9c1-f849918fa85c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:35.048883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a78efd-0cee-43f1-94d1-a5084d419964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:35.049988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a78efd-0cee-43f1-94d1-a5084d419964 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:35.050496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a78efd-0cee-43f1-94d1-a5084d419964 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:35.050973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a78efd-0cee-43f1-94d1-a5084d419964 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:35.051884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a78efd-0cee-43f1-94d1-a5084d419964 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:35.052392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a78efd-0cee-43f1-94d1-a5084d419964 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:35.054007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a78efd-0cee-43f1-94d1-a5084d419964 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:35.060452 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 479/959] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:35.065971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b2279-9ab7-41c8-9815-ee698855adb9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:35.113124 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:35.176896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b2279-9ab7-41c8-9815-ee698855adb9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:35.214024 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5247ee-4fff-42d3-9ac2-c2def1350738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:35.214699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5247ee-4fff-42d3-9ac2-c2def1350738 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:35.214986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5247ee-4fff-42d3-9ac2-c2def1350738 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:35.215269 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5247ee-4fff-42d3-9ac2-c2def1350738 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:35.215984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5247ee-4fff-42d3-9ac2-c2def1350738 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:35.216295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5247ee-4fff-42d3-9ac2-c2def1350738 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:35.217790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5247ee-4fff-42d3-9ac2-c2def1350738 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:35.229900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 481/960] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:36.397769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae4d09-395c-48e6-a489-f1e2c8dc8be1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:36.398768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae4d09-395c-48e6-a489-f1e2c8dc8be1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:36.399271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae4d09-395c-48e6-a489-f1e2c8dc8be1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:36.401165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 480/961] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:36.408213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9d66b-dc56-4a3a-9f3b-94f527649c1e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:36.409471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9d66b-dc56-4a3a-9f3b-94f527649c1e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:36.409993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9d66b-dc56-4a3a-9f3b-94f527649c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:36.429407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-34a9d66b-dc56-4a3a-9f3b-94f527649c1e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:36.430983 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-34a9d66b-dc56-4a3a-9f3b-94f527649c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:36.506301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 482/962] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:36.531958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae4d09-395c-48e6-a489-f1e2c8dc8be1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:36.632405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4feb17-4642-41df-8946-6078bf8d8c1f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:36.632731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4feb17-4642-41df-8946-6078bf8d8c1f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:36.632980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4feb17-4642-41df-8946-6078bf8d8c1f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:36.634702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 483/963] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:36.639456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae4d09-395c-48e6-a489-f1e2c8dc8be1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:36.640716 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc237d24-8248-4bf0-90f1-f782ead800a8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:36.641264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc237d24-8248-4bf0-90f1-f782ead800a8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:36.641554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc237d24-8248-4bf0-90f1-f782ead800a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:36.659720 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bc237d24-8248-4bf0-90f1-f782ead800a8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:36.660935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bc237d24-8248-4bf0-90f1-f782ead800a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:36.670733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:36.677397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9432d-0f81-401e-a863-a43ca7635a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:36.678176 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9432d-0f81-401e-a863-a43ca7635a39 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:36.678458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9432d-0f81-401e-a863-a43ca7635a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:36.678735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9432d-0f81-401e-a863-a43ca7635a39 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:36.679539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9432d-0f81-401e-a863-a43ca7635a39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:36.681075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9432d-0f81-401e-a863-a43ca7635a39 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:36.682648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: WARNING keystone.server.flask.common [None req-b9a9432d-0f81-401e-a863-a43ca7635a39 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 25 16:07:36.693473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:36.696833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.sql.core [None req-b9a9432d-0f81-401e-a863-a43ca7635a39 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 16:07:36.696833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 16:07:36.696833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [parameters: {'user_id': 'ddc7c36fbe524290b0ab537120d22a53', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 16:07:36.696833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80167) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b9a9432d-0f81-401e-a863-a43ca7635a39 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = self._query(query) May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application conn.query(q) May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result.read() May 25 16:07:36.707803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 16:07:36.708685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application next(self.gen) May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application yield resource May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application next(self.gen) May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.session.rollback() May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.force_reraise() May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise self.value May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application session.commit() May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 16:07:36.709621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._prepare_impl() May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.session.flush() May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._flush(objects) May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application compat.raise_( May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application flush_context.execute() May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rec.execute(self) May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 16:07:36.710549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = connection._execute_20( May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 16:07:36.711467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = self._query(query) May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application conn.query(q) May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result.read() May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 16:07:36.712306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ddc7c36fbe524290b0ab537120d22a53', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:36.713147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 25 16:07:36.720136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 25 16:07:36.722283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 25 16:07:36.722283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 25 16:07:36.722283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 16:07:36.722283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 16:07:36.722283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 16:07:36.722283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:36.722283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 481/964] 10.222.0.42 () {60 vars in 1131 bytes} [Wed May 25 16:07:36 2022] POST /identity/v3/users => generated 160 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:36.726961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9432d-0f81-401e-a863-a43ca7635a39 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:36.749791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 484/965] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:36.771957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc237d24-8248-4bf0-90f1-f782ead800a8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:36.866211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9432d-0f81-401e-a863-a43ca7635a39 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:36.908114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc946f-ae2a-470e-8540-809b02b3dfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:36.910313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc946f-ae2a-470e-8540-809b02b3dfae None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:36.910949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc946f-ae2a-470e-8540-809b02b3dfae None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:36.911573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc946f-ae2a-470e-8540-809b02b3dfae None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:36.913253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc237d24-8248-4bf0-90f1-f782ead800a8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:36.926426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc946f-ae2a-470e-8540-809b02b3dfae None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:36.928008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc946f-ae2a-470e-8540-809b02b3dfae None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e7bc946f-ae2a-470e-8540-809b02b3dfae None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:36.951682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:36.952888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:36.953875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:36.953875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:36.953875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:36.953875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:36.953875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:36.953875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:36.953875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:36.953875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:36.953875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 16:07:36.953875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:36.959815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 482/966] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:07:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:36.961363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc946f-ae2a-470e-8540-809b02b3dfae None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:36.973343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ac080-9c46-42b7-9d81-526a2d545110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:36.974263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ac080-9c46-42b7-9d81-526a2d545110 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:36.974564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ac080-9c46-42b7-9d81-526a2d545110 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:36.974905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ac080-9c46-42b7-9d81-526a2d545110 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:36.987489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ac080-9c46-42b7-9d81-526a2d545110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-513ac080-9c46-42b7-9d81-526a2d545110 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:36.996614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:36.997663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:36.998646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:37.002818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 485/967] 10.222.0.42 () {56 vars in 1120 bytes} [Wed May 25 16:07:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:37.004151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ac080-9c46-42b7-9d81-526a2d545110 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:37.031938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:37.034921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:37.086544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc946f-ae2a-470e-8540-809b02b3dfae None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:37.125056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ac080-9c46-42b7-9d81-526a2d545110 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:37.125671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a36a6-913b-4a4e-898f-b3f7c5e4b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:37.126392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a36a6-913b-4a4e-898f-b3f7c5e4b9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.126675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a36a6-913b-4a4e-898f-b3f7c5e4b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.126951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a36a6-913b-4a4e-898f-b3f7c5e4b9c3 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.127764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a36a6-913b-4a4e-898f-b3f7c5e4b9c3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:37.128123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a36a6-913b-4a4e-898f-b3f7c5e4b9c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:37.129596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a36a6-913b-4a4e-898f-b3f7c5e4b9c3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:37.163815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbac48e-9c4f-483b-ab23-0a7a9d573a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:37.164424 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbac48e-9c4f-483b-ab23-0a7a9d573a3d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.165131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbac48e-9c4f-483b-ab23-0a7a9d573a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.165131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbac48e-9c4f-483b-ab23-0a7a9d573a3d None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.165742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbac48e-9c4f-483b-ab23-0a7a9d573a3d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:37.166061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbac48e-9c4f-483b-ab23-0a7a9d573a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:37.167478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbac48e-9c4f-483b-ab23-0a7a9d573a3d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:37.181968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 483/968] 10.222.0.42 () {56 vars in 1101 bytes} [Wed May 25 16:07:36 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:37.181968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2108ec-9b74-4a95-9cf0-aa621a519718 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.181968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2108ec-9b74-4a95-9cf0-aa621a519718 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.181968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2108ec-9b74-4a95-9cf0-aa621a519718 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.181968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 484/969] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:37.181968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769febb-0942-48e3-afa6-11b400b874fb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.181968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769febb-0942-48e3-afa6-11b400b874fb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.182721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769febb-0942-48e3-afa6-11b400b874fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.187743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 486/970] 10.222.0.42 () {56 vars in 1115 bytes} [Wed May 25 16:07:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:37.194016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbac48e-9c4f-483b-ab23-0a7a9d573a3d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:37.201869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5769febb-0942-48e3-afa6-11b400b874fb None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:37.203438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5769febb-0942-48e3-afa6-11b400b874fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:37.297062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 485/971] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:37.314753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:37.314954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769febb-0942-48e3-afa6-11b400b874fb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:37.339234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbac48e-9c4f-483b-ab23-0a7a9d573a3d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:37.370158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:37.395069 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c4b0a-fa42-49a0-a4b2-1c8405846983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:37.396237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c4b0a-fa42-49a0-a4b2-1c8405846983 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.396582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c4b0a-fa42-49a0-a4b2-1c8405846983 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.396947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c4b0a-fa42-49a0-a4b2-1c8405846983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.410001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c4b0a-fa42-49a0-a4b2-1c8405846983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:37.411564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c4b0a-fa42-49a0-a4b2-1c8405846983 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-074c4b0a-fa42-49a0-a4b2-1c8405846983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:37.431516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:37.434932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:37.436422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:37.436422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:37.436422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:37.436422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:37.436422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 487/972] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:37.443192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c4b0a-fa42-49a0-a4b2-1c8405846983 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:37.457417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769febb-0942-48e3-afa6-11b400b874fb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:37.508709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212bd60-8de5-4cb4-b7ba-b55b7b5dd910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:37.510251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212bd60-8de5-4cb4-b7ba-b55b7b5dd910 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.511255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212bd60-8de5-4cb4-b7ba-b55b7b5dd910 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.511753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212bd60-8de5-4cb4-b7ba-b55b7b5dd910 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.512715 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212bd60-8de5-4cb4-b7ba-b55b7b5dd910 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:37.514409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212bd60-8de5-4cb4-b7ba-b55b7b5dd910 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c212bd60-8de5-4cb4-b7ba-b55b7b5dd910 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:37.532001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:37.533091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:37.534240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:37.534995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 486/973] 10.222.0.42 () {56 vars in 1126 bytes} [Wed May 25 16:07:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:37.541046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5145568-b5dd-4d06-9da1-45573aeea653 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.541340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5145568-b5dd-4d06-9da1-45573aeea653 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.542316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5145568-b5dd-4d06-9da1-45573aeea653 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.543690 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 487/974] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:07:37.544396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5145568-b5dd-4d06-9da1-45573aeea653 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:37.567263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c4b0a-fa42-49a0-a4b2-1c8405846983 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:37.602005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895f64e-bf86-4284-9b2f-ee5ba09196b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:37.602680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895f64e-bf86-4284-9b2f-ee5ba09196b7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.602961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895f64e-bf86-4284-9b2f-ee5ba09196b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.603235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895f64e-bf86-4284-9b2f-ee5ba09196b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.603990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895f64e-bf86-4284-9b2f-ee5ba09196b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:37.604829 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895f64e-bf86-4284-9b2f-ee5ba09196b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:37.606235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895f64e-bf86-4284-9b2f-ee5ba09196b7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:37.613375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:37.619351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 488/975] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:37.620655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18eb793-70ce-4fd7-9ab4-7d0a55b77b1b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.620941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18eb793-70ce-4fd7-9ab4-7d0a55b77b1b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.621211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18eb793-70ce-4fd7-9ab4-7d0a55b77b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.638444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c18eb793-70ce-4fd7-9ab4-7d0a55b77b1b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:37.639644 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c18eb793-70ce-4fd7-9ab4-7d0a55b77b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:37.649562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5145568-b5dd-4d06-9da1-45573aeea653 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:37.652694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:37.685186 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472385a1-bf85-4466-8e74-05c584c5dc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:37.685795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472385a1-bf85-4466-8e74-05c584c5dc51 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.686083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472385a1-bf85-4466-8e74-05c584c5dc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.686364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472385a1-bf85-4466-8e74-05c584c5dc51 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.687072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472385a1-bf85-4466-8e74-05c584c5dc51 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:37.687382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472385a1-bf85-4466-8e74-05c584c5dc51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:37.688754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472385a1-bf85-4466-8e74-05c584c5dc51 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:37.693702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 488/976] 10.222.0.42 () {56 vars in 1122 bytes} [Wed May 25 16:07:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:37.694511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472385a1-bf85-4466-8e74-05c584c5dc51 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:37.708443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 489/977] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:37.709282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18eb793-70ce-4fd7-9ab4-7d0a55b77b1b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:37.801171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472385a1-bf85-4466-8e74-05c584c5dc51 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:37.808197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18eb793-70ce-4fd7-9ab4-7d0a55b77b1b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:37.834338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:37.835810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0aaac6-d305-4b2d-89fa-d5d92ee74c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:37.836751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aaac6-d305-4b2d-89fa-d5d92ee74c6f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.837064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aaac6-d305-4b2d-89fa-d5d92ee74c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.837369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aaac6-d305-4b2d-89fa-d5d92ee74c6f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.843243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd26ed7-3af7-4734-b4f2-944b807e2a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:37.843916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd26ed7-3af7-4734-b4f2-944b807e2a6a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.844209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd26ed7-3af7-4734-b4f2-944b807e2a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.844486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd26ed7-3af7-4734-b4f2-944b807e2a6a None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.845185 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0aaac6-d305-4b2d-89fa-d5d92ee74c6f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:37.845331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd26ed7-3af7-4734-b4f2-944b807e2a6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:37.845545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd26ed7-3af7-4734-b4f2-944b807e2a6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:37.846634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0aaac6-d305-4b2d-89fa-d5d92ee74c6f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:37.846952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd26ed7-3af7-4734-b4f2-944b807e2a6a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:37.851840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 490/978] 10.222.0.42 () {56 vars in 1122 bytes} [Wed May 25 16:07:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:37.852602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd26ed7-3af7-4734-b4f2-944b807e2a6a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4d0aaac6-d305-4b2d-89fa-d5d92ee74c6f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:37.866480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:37.867478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:37.868423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:37.868423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:37.868423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:37.868423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:37.868423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:37.868423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:37.868423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:37.868423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:37.868423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 16:07:37.868423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:37.868423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 489/979] 10.222.0.42 () {56 vars in 1117 bytes} [Wed May 25 16:07:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:37.869149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0aaac6-d305-4b2d-89fa-d5d92ee74c6f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:37.898449 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:37.957907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd26ed7-3af7-4734-b4f2-944b807e2a6a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:37.980058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0aaac6-d305-4b2d-89fa-d5d92ee74c6f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:37.997019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed94e53-aa5c-4a04-b952-216a995cb6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:37.997686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94e53-aa5c-4a04-b952-216a995cb6af None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:37.997998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94e53-aa5c-4a04-b952-216a995cb6af None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:37.998275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94e53-aa5c-4a04-b952-216a995cb6af None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:37.998993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed94e53-aa5c-4a04-b952-216a995cb6af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:38.000365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed94e53-aa5c-4a04-b952-216a995cb6af None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:38.015166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3338ad-f621-4370-92d5-db54d23e393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:38.015802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 491/980] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:07:37 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:38.015947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3338ad-f621-4370-92d5-db54d23e393c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:38.016113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3338ad-f621-4370-92d5-db54d23e393c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:38.016372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3338ad-f621-4370-92d5-db54d23e393c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:38.016768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed94e53-aa5c-4a04-b952-216a995cb6af None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:38.017551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3338ad-f621-4370-92d5-db54d23e393c None admin] RBAC: Adding query filter params (interface=admin, service_id=9cabd0daf45a401caca2a9f1da7b19fe, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:38.017662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3338ad-f621-4370-92d5-db54d23e393c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:38.019037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3338ad-f621-4370-92d5-db54d23e393c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:38.025255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 490/981] 10.222.0.42 () {56 vars in 1249 bytes} [Wed May 25 16:07:37 2022] GET /identity/v3/endpoints?service_id=9cabd0daf45a401caca2a9f1da7b19fe&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:38.031773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3338ad-f621-4370-92d5-db54d23e393c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:38.100819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:38.148286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:38.162257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3338ad-f621-4370-92d5-db54d23e393c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:38.166735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed94e53-aa5c-4a04-b952-216a995cb6af None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:38.205010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4bec5-003b-4faa-b055-a0d4741211fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:38.205687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4bec5-003b-4faa-b055-a0d4741211fb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:38.206758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3792f3f-0bf4-4ad6-8943-596c77922085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:38.206874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4bec5-003b-4faa-b055-a0d4741211fb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:38.206874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4bec5-003b-4faa-b055-a0d4741211fb None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:38.207009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3792f3f-0bf4-4ad6-8943-596c77922085 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:38.207009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3792f3f-0bf4-4ad6-8943-596c77922085 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:38.207144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4bec5-003b-4faa-b055-a0d4741211fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:38.207216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3792f3f-0bf4-4ad6-8943-596c77922085 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:38.207975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3792f3f-0bf4-4ad6-8943-596c77922085 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:38.208233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3792f3f-0bf4-4ad6-8943-596c77922085 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:38.208331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4bec5-003b-4faa-b055-a0d4741211fb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:38.210045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3792f3f-0bf4-4ad6-8943-596c77922085 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:38.215178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 491/982] 10.222.0.42 () {56 vars in 1087 bytes} [Wed May 25 16:07:38 2022] GET /identity/v3/services => generated 1727 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:07:38.233268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 492/983] 10.222.0.42 () {56 vars in 1149 bytes} [Wed May 25 16:07:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:38.239534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4bec5-003b-4faa-b055-a0d4741211fb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:38.361697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4bec5-003b-4faa-b055-a0d4741211fb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:38.399236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e51bc-7496-49b7-b948-71088b3a51de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:38.399904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e51bc-7496-49b7-b948-71088b3a51de None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:38.400193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e51bc-7496-49b7-b948-71088b3a51de None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:38.400464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e51bc-7496-49b7-b948-71088b3a51de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:38.435569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:38.480056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e51bc-7496-49b7-b948-71088b3a51de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:38.481432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e51bc-7496-49b7-b948-71088b3a51de None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:38.594956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 492/984] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:38 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:07:38.596229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3792f3f-0bf4-4ad6-8943-596c77922085 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:38.708287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3792f3f-0bf4-4ad6-8943-596c77922085 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:38.716620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:38.746934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d502b0d-8001-44a1-bdf6-4321266a5a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:38.747657 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d502b0d-8001-44a1-bdf6-4321266a5a26 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:38.747941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d502b0d-8001-44a1-bdf6-4321266a5a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:38.748217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d502b0d-8001-44a1-bdf6-4321266a5a26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:38.758579 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d502b0d-8001-44a1-bdf6-4321266a5a26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:38.760037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d502b0d-8001-44a1-bdf6-4321266a5a26 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5d502b0d-8001-44a1-bdf6-4321266a5a26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:38.778901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:38.779859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:38.780750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:38.780750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:38.780750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:38.780750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:38.780750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 493/985] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:38.785972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e51bc-7496-49b7-b948-71088b3a51de None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:38.844941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:38.889918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e51bc-7496-49b7-b948-71088b3a51de None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:38.927872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7a6e0-5f89-417e-b068-335daa387de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:38.928550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7a6e0-5f89-417e-b068-335daa387de7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:38.928833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7a6e0-5f89-417e-b068-335daa387de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:38.929111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7a6e0-5f89-417e-b068-335daa387de7 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:38.929861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7a6e0-5f89-417e-b068-335daa387de7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:38.930170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7a6e0-5f89-417e-b068-335daa387de7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:38.931535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7a6e0-5f89-417e-b068-335daa387de7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:38.943738 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 493/986] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:38.949675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d502b0d-8001-44a1-bdf6-4321266a5a26 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:39.069441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d502b0d-8001-44a1-bdf6-4321266a5a26 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:39.106975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c91266-85d8-48c0-a96d-8c64ae5f4cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:39.107776 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c91266-85d8-48c0-a96d-8c64ae5f4cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:39.108060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c91266-85d8-48c0-a96d-8c64ae5f4cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:39.108336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c91266-85d8-48c0-a96d-8c64ae5f4cc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:39.116130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c91266-85d8-48c0-a96d-8c64ae5f4cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:39.117571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c91266-85d8-48c0-a96d-8c64ae5f4cc7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-76c91266-85d8-48c0-a96d-8c64ae5f4cc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:39.131377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:39.132278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:39.133203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:39.133203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:39.133203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:39.133203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:39.133203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:39.133203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:39.133203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:39.133203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:39.137615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 494/987] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:39.138865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7a6e0-5f89-417e-b068-335daa387de7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:39.243033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7a6e0-5f89-417e-b068-335daa387de7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:39.262759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:39.282140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee35b8-0041-40fc-83a1-fea899f471e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:39.282805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee35b8-0041-40fc-83a1-fea899f471e0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:39.283088 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee35b8-0041-40fc-83a1-fea899f471e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:39.283363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee35b8-0041-40fc-83a1-fea899f471e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:39.284662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee35b8-0041-40fc-83a1-fea899f471e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:39.284774 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee35b8-0041-40fc-83a1-fea899f471e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:39.286153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee35b8-0041-40fc-83a1-fea899f471e0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:39.300453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 494/988] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:39.305371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c91266-85d8-48c0-a96d-8c64ae5f4cc7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:39.327343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:39.414449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c91266-85d8-48c0-a96d-8c64ae5f4cc7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:39.459203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e627dc-1d60-4617-84f4-fde3954fadf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:39.459864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e627dc-1d60-4617-84f4-fde3954fadf9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:39.460147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e627dc-1d60-4617-84f4-fde3954fadf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:39.460424 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e627dc-1d60-4617-84f4-fde3954fadf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:39.461254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e627dc-1d60-4617-84f4-fde3954fadf9 None admin] RBAC: Adding query filter params (role.id=73f22629793d47f086feb1e90de17023, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=730cb89f7a214fde9c5040674323f8db) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:39.461569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e627dc-1d60-4617-84f4-fde3954fadf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:39.463029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e627dc-1d60-4617-84f4-fde3954fadf9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:39.477761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 495/989] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:39 2022] GET /identity/v3/role_assignments?user.id=730cb89f7a214fde9c5040674323f8db&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=73f22629793d47f086feb1e90de17023 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:40.367337 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb11541-64ff-4937-b378-45d2771236bd None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:40.367895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb11541-64ff-4937-b378-45d2771236bd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:40.367895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb11541-64ff-4937-b378-45d2771236bd None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:40.369300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 495/990] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:40.374755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a8893-f168-486a-82b8-96c631e4a341 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:40.375158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a8893-f168-486a-82b8-96c631e4a341 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:40.375343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a8893-f168-486a-82b8-96c631e4a341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:40.392888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b61a8893-f168-486a-82b8-96c631e4a341 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:40.394107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b61a8893-f168-486a-82b8-96c631e4a341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:40.473113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 496/991] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:40.497196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb11541-64ff-4937-b378-45d2771236bd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:40.499241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8fe10-3079-44a4-ae20-dbc03b7eff6c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:40.499528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8fe10-3079-44a4-ae20-dbc03b7eff6c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:40.499796 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8fe10-3079-44a4-ae20-dbc03b7eff6c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:40.501316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 497/992] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:07:40.506135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad6bbb-4e1d-4489-af69-6e67a32e5c78 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:40.506421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad6bbb-4e1d-4489-af69-6e67a32e5c78 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:40.506697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad6bbb-4e1d-4489-af69-6e67a32e5c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:40.528789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-55ad6bbb-4e1d-4489-af69-6e67a32e5c78 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:40.530001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-55ad6bbb-4e1d-4489-af69-6e67a32e5c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:40.538350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:40.606845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 498/993] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:40.621233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb11541-64ff-4937-b378-45d2771236bd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:40.630031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad6bbb-4e1d-4489-af69-6e67a32e5c78 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:40.659537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4c1b3-d994-4bb0-9d27-0e0c2d0ab9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:40.660380 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4c1b3-d994-4bb0-9d27-0e0c2d0ab9cd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:40.660671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4c1b3-d994-4bb0-9d27-0e0c2d0ab9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:40.660968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4c1b3-d994-4bb0-9d27-0e0c2d0ab9cd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:40.661744 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4c1b3-d994-4bb0-9d27-0e0c2d0ab9cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:40.663160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4c1b3-d994-4bb0-9d27-0e0c2d0ab9cd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b3f4c1b3-d994-4bb0-9d27-0e0c2d0ab9cd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:40.676193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:40.677262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 16:07:40.678144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:40.681878 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 496/994] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:40.683645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4c1b3-d994-4bb0-9d27-0e0c2d0ab9cd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:40.735233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:40.739794 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad6bbb-4e1d-4489-af69-6e67a32e5c78 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:40.774286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e886e15-fa6b-4ff4-ad86-ff0da9c093f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:40.775088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e886e15-fa6b-4ff4-ad86-ff0da9c093f3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:40.775372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e886e15-fa6b-4ff4-ad86-ff0da9c093f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:40.776157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e886e15-fa6b-4ff4-ad86-ff0da9c093f3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:40.776842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e886e15-fa6b-4ff4-ad86-ff0da9c093f3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:40.778254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e886e15-fa6b-4ff4-ad86-ff0da9c093f3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7e886e15-fa6b-4ff4-ad86-ff0da9c093f3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:40.795028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:40.796077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 16:07:40.796925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:40.798214 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 499/995] 10.222.0.42 () {56 vars in 1126 bytes} [Wed May 25 16:07:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:40.802792 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e886e15-fa6b-4ff4-ad86-ff0da9c093f3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:40.805602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4c1b3-d994-4bb0-9d27-0e0c2d0ab9cd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:40.812772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:40.844957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc12fe0-2427-452f-80ca-a156b94f5d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:40.845632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc12fe0-2427-452f-80ca-a156b94f5d06 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:40.845944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc12fe0-2427-452f-80ca-a156b94f5d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:40.846642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc12fe0-2427-452f-80ca-a156b94f5d06 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:40.847354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc12fe0-2427-452f-80ca-a156b94f5d06 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:40.847664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc12fe0-2427-452f-80ca-a156b94f5d06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:40.850055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc12fe0-2427-452f-80ca-a156b94f5d06 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:40.856732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 497/996] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:40.862208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc12fe0-2427-452f-80ca-a156b94f5d06 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:40.912278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e886e15-fa6b-4ff4-ad86-ff0da9c093f3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:40.949697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb2b42-87de-449d-ad68-6e5ee261e624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:40.950570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb2b42-87de-449d-ad68-6e5ee261e624 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:40.950868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb2b42-87de-449d-ad68-6e5ee261e624 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:40.951147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb2b42-87de-449d-ad68-6e5ee261e624 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:40.951863 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb2b42-87de-449d-ad68-6e5ee261e624 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:40.952175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb2b42-87de-449d-ad68-6e5ee261e624 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:40.953553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb2b42-87de-449d-ad68-6e5ee261e624 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:40.958642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 500/997] 10.222.0.42 () {56 vars in 1122 bytes} [Wed May 25 16:07:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:40.964816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb2b42-87de-449d-ad68-6e5ee261e624 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:40.972237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc12fe0-2427-452f-80ca-a156b94f5d06 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:40.975699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:41.014573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795dc22-a1af-4822-ba77-d22bc6648d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:41.015406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795dc22-a1af-4822-ba77-d22bc6648d4f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:41.015608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795dc22-a1af-4822-ba77-d22bc6648d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:41.015754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795dc22-a1af-4822-ba77-d22bc6648d4f None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:41.016469 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795dc22-a1af-4822-ba77-d22bc6648d4f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:41.016766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795dc22-a1af-4822-ba77-d22bc6648d4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:41.018167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795dc22-a1af-4822-ba77-d22bc6648d4f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:41.024331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 498/998] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:41.030213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795dc22-a1af-4822-ba77-d22bc6648d4f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:41.044289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:41.078013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb2b42-87de-449d-ad68-6e5ee261e624 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:41.114729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb140bd-26df-4032-adb0-1681de82a2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:41.115796 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb140bd-26df-4032-adb0-1681de82a2ba None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:41.116306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb140bd-26df-4032-adb0-1681de82a2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:41.116797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb140bd-26df-4032-adb0-1681de82a2ba None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:41.117631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb140bd-26df-4032-adb0-1681de82a2ba None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:41.117990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb140bd-26df-4032-adb0-1681de82a2ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:41.119370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb140bd-26df-4032-adb0-1681de82a2ba None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:41.131507 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 501/999] 10.222.0.42 () {56 vars in 1122 bytes} [Wed May 25 16:07:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:41.133841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb140bd-26df-4032-adb0-1681de82a2ba None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:41.148671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795dc22-a1af-4822-ba77-d22bc6648d4f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:41.193550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0e751-2b57-4a81-8d6a-620142b1bfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:41.194161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e751-2b57-4a81-8d6a-620142b1bfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:41.194758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e751-2b57-4a81-8d6a-620142b1bfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:41.194967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e751-2b57-4a81-8d6a-620142b1bfc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:41.196306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0e751-2b57-4a81-8d6a-620142b1bfc1 None admin] RBAC: Adding query filter params (interface=public, service_id=ae3e4137103a441ea18804a2052266a4, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:41.196762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0e751-2b57-4a81-8d6a-620142b1bfc1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:41.199076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0e751-2b57-4a81-8d6a-620142b1bfc1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:41.204484 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 499/1000] 10.222.0.42 () {56 vars in 1251 bytes} [Wed May 25 16:07:41 2022] GET /identity/v3/endpoints?service_id=ae3e4137103a441ea18804a2052266a4&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:41.210181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0e751-2b57-4a81-8d6a-620142b1bfc1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:41.242739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:41.244419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb140bd-26df-4032-adb0-1681de82a2ba None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:41.280590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dda24a-2bc3-44ff-a201-cf1ae592b7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:41.281258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dda24a-2bc3-44ff-a201-cf1ae592b7cc None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:41.281546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dda24a-2bc3-44ff-a201-cf1ae592b7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:41.281851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dda24a-2bc3-44ff-a201-cf1ae592b7cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:41.282589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dda24a-2bc3-44ff-a201-cf1ae592b7cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:41.284407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dda24a-2bc3-44ff-a201-cf1ae592b7cc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:41.287988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:41.309771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 502/1001] 10.222.0.42 () {60 vars in 1144 bytes} [Wed May 25 16:07:41 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:07:41.334864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0e751-2b57-4a81-8d6a-620142b1bfc1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:41.382777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44549bfd-e6e6-4ade-bf08-f73abc9f7b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:41.383592 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44549bfd-e6e6-4ade-bf08-f73abc9f7b6c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:41.383844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44549bfd-e6e6-4ade-bf08-f73abc9f7b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:41.384128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44549bfd-e6e6-4ade-bf08-f73abc9f7b6c None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:41.384879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44549bfd-e6e6-4ade-bf08-f73abc9f7b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:41.386948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44549bfd-e6e6-4ade-bf08-f73abc9f7b6c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:41.397768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 500/1002] 10.222.0.42 () {56 vars in 1087 bytes} [Wed May 25 16:07:41 2022] GET /identity/v3/services => generated 1727 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:41.751087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242ac6-4717-4ec1-9ae6-f254ab53e0ab None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:41.751656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242ac6-4717-4ec1-9ae6-f254ab53e0ab None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:41.751656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242ac6-4717-4ec1-9ae6-f254ab53e0ab None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:41.753159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 503/1003] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:41.758205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290ed5c-6c61-4aca-a0a2-5e10f2248994 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:41.758497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290ed5c-6c61-4aca-a0a2-5e10f2248994 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:41.758776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290ed5c-6c61-4aca-a0a2-5e10f2248994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:41.776452 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f290ed5c-6c61-4aca-a0a2-5e10f2248994 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:41.777639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f290ed5c-6c61-4aca-a0a2-5e10f2248994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:41.847181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 501/1004] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:41 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:41.870425 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1242ac6-4717-4ec1-9ae6-f254ab53e0ab None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:41.978078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1242ac6-4717-4ec1-9ae6-f254ab53e0ab None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:42.013302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9dbfc5-5d8f-45a8-8ceb-c7d88146da58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:42.014044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9dbfc5-5d8f-45a8-8ceb-c7d88146da58 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:42.014331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9dbfc5-5d8f-45a8-8ceb-c7d88146da58 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:42.014604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9dbfc5-5d8f-45a8-8ceb-c7d88146da58 None admin] PATH_INFO: `/v3/services/image` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:42.015255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9dbfc5-5d8f-45a8-8ceb-c7d88146da58 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:42.016625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9dbfc5-5d8f-45a8-8ceb-c7d88146da58 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-cc9dbfc5-5d8f-45a8-8ceb-c7d88146da58 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:42.029039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:42.030094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 16:07:42.030984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:42.032568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 504/1005] 10.222.0.42 () {56 vars in 1105 bytes} [Wed May 25 16:07:41 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:42.036530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290ed5c-6c61-4aca-a0a2-5e10f2248994 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:42.063153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:42.144211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290ed5c-6c61-4aca-a0a2-5e10f2248994 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:42.184077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79187b4-3a18-4e4a-86c9-92e833f66d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:42.184761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79187b4-3a18-4e4a-86c9-92e833f66d13 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:42.185174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79187b4-3a18-4e4a-86c9-92e833f66d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:42.185493 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79187b4-3a18-4e4a-86c9-92e833f66d13 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:42.186335 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79187b4-3a18-4e4a-86c9-92e833f66d13 None admin] RBAC: Adding query filter params (name=image) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:42.186664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79187b4-3a18-4e4a-86c9-92e833f66d13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:42.188138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79187b4-3a18-4e4a-86c9-92e833f66d13 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:42.200136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 502/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:42.201702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9dbfc5-5d8f-45a8-8ceb-c7d88146da58 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:42.262784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:42.308157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9dbfc5-5d8f-45a8-8ceb-c7d88146da58 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:42.341372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded677c7-0fb9-48cf-8796-94ebad95e50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:42.342062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded677c7-0fb9-48cf-8796-94ebad95e50b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:42.342348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded677c7-0fb9-48cf-8796-94ebad95e50b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:42.342624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded677c7-0fb9-48cf-8796-94ebad95e50b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:42.343329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded677c7-0fb9-48cf-8796-94ebad95e50b None admin] RBAC: Adding query filter params (type=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:42.343635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded677c7-0fb9-48cf-8796-94ebad95e50b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:42.344999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded677c7-0fb9-48cf-8796-94ebad95e50b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:42.354288 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 505/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:42 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:43.601201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2fbcf-9cdd-416b-8046-c7b8fea240dd None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:43.601887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2fbcf-9cdd-416b-8046-c7b8fea240dd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:43.601887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2fbcf-9cdd-416b-8046-c7b8fea240dd None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:43.603368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 503/1008] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:07:43.609080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d1f15-0d67-4fad-9209-844c86ab28b5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:43.609538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d1f15-0d67-4fad-9209-844c86ab28b5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:43.609756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d1f15-0d67-4fad-9209-844c86ab28b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:43.630344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1f5d1f15-0d67-4fad-9209-844c86ab28b5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:43.632089 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1f5d1f15-0d67-4fad-9209-844c86ab28b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:43.679249 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858c79f-78de-44e3-99b2-780866da2c88 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:43.679572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858c79f-78de-44e3-99b2-780866da2c88 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:43.679815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858c79f-78de-44e3-99b2-780866da2c88 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:43.681331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 504/1009] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:07:43.687048 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3385de-389f-4cab-bb4d-2dc32b787b76 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:43.687265 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3385de-389f-4cab-bb4d-2dc32b787b76 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:43.687545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3385de-389f-4cab-bb4d-2dc32b787b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:43.704671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ef3385de-389f-4cab-bb4d-2dc32b787b76 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:43.705908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ef3385de-389f-4cab-bb4d-2dc32b787b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:43.714143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 506/1010] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:43 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:43.732918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d1f15-0d67-4fad-9209-844c86ab28b5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:43.780886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 505/1011] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:43 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:43.790086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:43.804124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3385de-389f-4cab-bb4d-2dc32b787b76 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:43.823130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:43.842854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d1f15-0d67-4fad-9209-844c86ab28b5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:43.879285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24d496-1f91-4e5d-bd23-4f82ae33bb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:43.880084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24d496-1f91-4e5d-bd23-4f82ae33bb0b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:43.880362 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24d496-1f91-4e5d-bd23-4f82ae33bb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:43.880640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24d496-1f91-4e5d-bd23-4f82ae33bb0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:43.881840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24d496-1f91-4e5d-bd23-4f82ae33bb0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:43.883291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24d496-1f91-4e5d-bd23-4f82ae33bb0b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:43.884979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 507/1012] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:43.891811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24d496-1f91-4e5d-bd23-4f82ae33bb0b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:43.918304 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3385de-389f-4cab-bb4d-2dc32b787b76 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:43.959568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbeb80-42db-4054-bb18-426cd786b9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:43.960292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbeb80-42db-4054-bb18-426cd786b9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:43.960579 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbeb80-42db-4054-bb18-426cd786b9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:43.960854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbeb80-42db-4054-bb18-426cd786b9f1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:43.961501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbeb80-42db-4054-bb18-426cd786b9f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:43.962921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbeb80-42db-4054-bb18-426cd786b9f1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-bebbeb80-42db-4054-bb18-426cd786b9f1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:43.974966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:43.976018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 16:07:43.976872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:43.981979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 506/1013] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:43.982809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbeb80-42db-4054-bb18-426cd786b9f1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:44.008749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24d496-1f91-4e5d-bd23-4f82ae33bb0b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:44.030742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:44.049134 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb2742-9a92-4613-b507-ea3efff6f455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:44.049847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb2742-9a92-4613-b507-ea3efff6f455 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:44.050194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb2742-9a92-4613-b507-ea3efff6f455 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:44.050407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb2742-9a92-4613-b507-ea3efff6f455 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:44.051222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb2742-9a92-4613-b507-ea3efff6f455 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:44.052551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb2742-9a92-4613-b507-ea3efff6f455 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:44.072429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:44.084295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 508/1014] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:07:43 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:07:44.105597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbeb80-42db-4054-bb18-426cd786b9f1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:44.145387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c7c1e-1b51-46e6-92db-b0e135ca645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:44.146112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c7c1e-1b51-46e6-92db-b0e135ca645c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:44.146398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c7c1e-1b51-46e6-92db-b0e135ca645c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:44.146673 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c7c1e-1b51-46e6-92db-b0e135ca645c None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:44.147402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c7c1e-1b51-46e6-92db-b0e135ca645c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:44.147710 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c7c1e-1b51-46e6-92db-b0e135ca645c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:44.149134 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c7c1e-1b51-46e6-92db-b0e135ca645c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:44.160908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 507/1015] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:44.162631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb2742-9a92-4613-b507-ea3efff6f455 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:44.280732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb2742-9a92-4613-b507-ea3efff6f455 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:44.319168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada038c9-8447-4cc8-91c7-7b8aa773a512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:44.319843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada038c9-8447-4cc8-91c7-7b8aa773a512 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:44.320135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada038c9-8447-4cc8-91c7-7b8aa773a512 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:44.320409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada038c9-8447-4cc8-91c7-7b8aa773a512 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:44.321122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada038c9-8447-4cc8-91c7-7b8aa773a512 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:44.321431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada038c9-8447-4cc8-91c7-7b8aa773a512 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:44.322920 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada038c9-8447-4cc8-91c7-7b8aa773a512 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:44.328328 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 509/1016] 10.222.0.42 () {56 vars in 1112 bytes} [Wed May 25 16:07:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:44.333983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c7c1e-1b51-46e6-92db-b0e135ca645c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:44.452064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c7c1e-1b51-46e6-92db-b0e135ca645c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:44.491285 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d728b0-5797-4ec9-ada9-7570dc51909f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:44.491953 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d728b0-5797-4ec9-ada9-7570dc51909f None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:44.492670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d728b0-5797-4ec9-ada9-7570dc51909f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:44.492957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d728b0-5797-4ec9-ada9-7570dc51909f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:44.493727 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d728b0-5797-4ec9-ada9-7570dc51909f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:44.496377 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d728b0-5797-4ec9-ada9-7570dc51909f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:44.513401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:44.521167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 508/1017] 10.222.0.42 () {60 vars in 1144 bytes} [Wed May 25 16:07:44 2022] POST /identity/v3/endpoints => generated 334 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:44.646164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cf5d2-16ca-402b-b93e-369ff73a6393 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:44.646700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cf5d2-16ca-402b-b93e-369ff73a6393 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:44.646778 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cf5d2-16ca-402b-b93e-369ff73a6393 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:44.648495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 510/1018] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:44.655007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c828f-3e62-4048-a6c4-9e368f530eb8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:44.655546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c828f-3e62-4048-a6c4-9e368f530eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:44.655546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c828f-3e62-4048-a6c4-9e368f530eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:44.674358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-851c828f-3e62-4048-a6c4-9e368f530eb8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:44.675574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-851c828f-3e62-4048-a6c4-9e368f530eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:44.758973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 509/1019] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:44 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:44.782969 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cf5d2-16ca-402b-b93e-369ff73a6393 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:44.821486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:44.893162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cf5d2-16ca-402b-b93e-369ff73a6393 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:44.928548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a7074-bb98-49e0-b6bf-e4bff05ded92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:44.929125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a7074-bb98-49e0-b6bf-e4bff05ded92 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:44.929407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a7074-bb98-49e0-b6bf-e4bff05ded92 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:44.929686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a7074-bb98-49e0-b6bf-e4bff05ded92 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:44.930441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a7074-bb98-49e0-b6bf-e4bff05ded92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:44.931916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a7074-bb98-49e0-b6bf-e4bff05ded92 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:44.947810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 511/1020] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:07:44 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:07:46.405422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a0e92-816e-4a5e-97e0-088a0d40476a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.406044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a0e92-816e-4a5e-97e0-088a0d40476a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.406044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a0e92-816e-4a5e-97e0-088a0d40476a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.407529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 510/1021] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:46.413450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a1819-4728-412b-b937-fd17afa21f88 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.413952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a1819-4728-412b-b937-fd17afa21f88 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.414058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a1819-4728-412b-b937-fd17afa21f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.434340 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5d7a1819-4728-412b-b937-fd17afa21f88 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:46.435910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5d7a1819-4728-412b-b937-fd17afa21f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:46.511590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 512/1022] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:46.536676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a0e92-816e-4a5e-97e0-088a0d40476a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.645909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a0e92-816e-4a5e-97e0-088a0d40476a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.676823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:46.688046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146f131-12c0-4d0f-a293-b0c53d14cc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.688877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146f131-12c0-4d0f-a293-b0c53d14cc47 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.689163 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146f131-12c0-4d0f-a293-b0c53d14cc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.689603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146f131-12c0-4d0f-a293-b0c53d14cc47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.690939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146f131-12c0-4d0f-a293-b0c53d14cc47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.692395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146f131-12c0-4d0f-a293-b0c53d14cc47 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.698977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 511/1023] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:46.700263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a1819-4728-412b-b937-fd17afa21f88 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.725219 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:46.805730 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a1819-4728-412b-b937-fd17afa21f88 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.841594 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb48323-8923-4fbb-85d6-b738d0263476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.842297 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48323-8923-4fbb-85d6-b738d0263476 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.842577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48323-8923-4fbb-85d6-b738d0263476 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.842849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48323-8923-4fbb-85d6-b738d0263476 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.843562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb48323-8923-4fbb-85d6-b738d0263476 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.844989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb48323-8923-4fbb-85d6-b738d0263476 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.859608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c32bf-6195-4d84-9962-15127f582ce5 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.859892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c32bf-6195-4d84-9962-15127f582ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.860189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c32bf-6195-4d84-9962-15127f582ce5 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.861537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 512/1024] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:07:46.866638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41735411-0719-45b0-85d0-492265714ea1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.866919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41735411-0719-45b0-85d0-492265714ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.867192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41735411-0719-45b0-85d0-492265714ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.872084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 513/1025] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:46.887461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-41735411-0719-45b0-85d0-492265714ea1 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:46.888839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-41735411-0719-45b0-85d0-492265714ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:46.966635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 513/1026] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:46.990694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb48323-8923-4fbb-85d6-b738d0263476 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.095336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:47.121227 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb48323-8923-4fbb-85d6-b738d0263476 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.158112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548322e-524f-4b3a-b68c-63d013d3014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.159216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548322e-524f-4b3a-b68c-63d013d3014c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.159532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548322e-524f-4b3a-b68c-63d013d3014c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.159840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548322e-524f-4b3a-b68c-63d013d3014c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.173165 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548322e-524f-4b3a-b68c-63d013d3014c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3548322e-524f-4b3a-b68c-63d013d3014c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:47.184445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:47.185466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:47.186435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 514/1027] 10.222.0.42 () {56 vars in 1120 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:47.191766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41735411-0719-45b0-85d0-492265714ea1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.276109 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e52a6-d573-4dd0-89b4-51aa543665ee None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.276416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e52a6-d573-4dd0-89b4-51aa543665ee None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.276619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e52a6-d573-4dd0-89b4-51aa543665ee None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.278164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 515/1028] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:07:47.283252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938e186-cfb5-495b-9111-2b30208ccb40 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.283799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938e186-cfb5-495b-9111-2b30208ccb40 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.284081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938e186-cfb5-495b-9111-2b30208ccb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.300261 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41735411-0719-45b0-85d0-492265714ea1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.302211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0938e186-cfb5-495b-9111-2b30208ccb40 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:47.303476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0938e186-cfb5-495b-9111-2b30208ccb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:47.326799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:47.338953 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e0f4e-16af-4558-bc9b-e6277a4a67cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.339864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e0f4e-16af-4558-bc9b-e6277a4a67cf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.340207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e0f4e-16af-4558-bc9b-e6277a4a67cf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.340416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e0f4e-16af-4558-bc9b-e6277a4a67cf None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.341164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e0f4e-16af-4558-bc9b-e6277a4a67cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:47.341471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e0f4e-16af-4558-bc9b-e6277a4a67cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.343313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e0f4e-16af-4558-bc9b-e6277a4a67cf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.352374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 514/1029] 10.222.0.42 () {56 vars in 1115 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:47.363153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e0f4e-16af-4558-bc9b-e6277a4a67cf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.392442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 516/1030] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:47.415787 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938e186-cfb5-495b-9111-2b30208ccb40 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.469146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e0f4e-16af-4558-bc9b-e6277a4a67cf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.495587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:47.505234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432878ef-0eb3-45f4-8b24-d47b12e5a084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.506082 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432878ef-0eb3-45f4-8b24-d47b12e5a084 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.506362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432878ef-0eb3-45f4-8b24-d47b12e5a084 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.506634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432878ef-0eb3-45f4-8b24-d47b12e5a084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.518317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432878ef-0eb3-45f4-8b24-d47b12e5a084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.519767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432878ef-0eb3-45f4-8b24-d47b12e5a084 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.535321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938e186-cfb5-495b-9111-2b30208ccb40 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-432878ef-0eb3-45f4-8b24-d47b12e5a084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:47.538178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:47.539224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:47.540123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:47.540123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:47.540123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:47.540123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:47.540123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 515/1031] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:47.545055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432878ef-0eb3-45f4-8b24-d47b12e5a084 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.576447 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ea0f3-0aed-4b2e-8166-7506cb6f23c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.577350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ea0f3-0aed-4b2e-8166-7506cb6f23c8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.577632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ea0f3-0aed-4b2e-8166-7506cb6f23c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.577948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ea0f3-0aed-4b2e-8166-7506cb6f23c8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.578685 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ea0f3-0aed-4b2e-8166-7506cb6f23c8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.579984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ea0f3-0aed-4b2e-8166-7506cb6f23c8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.589910 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-973ea0f3-0aed-4b2e-8166-7506cb6f23c8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:47.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:47.593106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:47.594027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 517/1032] 10.222.0.42 () {56 vars in 1105 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:47.599959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ea0f3-0aed-4b2e-8166-7506cb6f23c8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.663549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432878ef-0eb3-45f4-8b24-d47b12e5a084 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.701404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b7623-408b-47ea-9d7c-286a5dd441f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.702084 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b7623-408b-47ea-9d7c-286a5dd441f3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.702365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b7623-408b-47ea-9d7c-286a5dd441f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.702735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b7623-408b-47ea-9d7c-286a5dd441f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.703359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b7623-408b-47ea-9d7c-286a5dd441f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:47.703666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b7623-408b-47ea-9d7c-286a5dd441f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.705040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b7623-408b-47ea-9d7c-286a5dd441f3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.711048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ea0f3-0aed-4b2e-8166-7506cb6f23c8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.723035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 516/1033] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:47.724145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b7623-408b-47ea-9d7c-286a5dd441f3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.747430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:47.753488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0dced-a5a9-4ab0-8a7d-a70e1caa5885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.754307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0dced-a5a9-4ab0-8a7d-a70e1caa5885 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.754656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0dced-a5a9-4ab0-8a7d-a70e1caa5885 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.754999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0dced-a5a9-4ab0-8a7d-a70e1caa5885 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.755768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0dced-a5a9-4ab0-8a7d-a70e1caa5885 None admin] RBAC: Adding query filter params (name=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:47.756078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0dced-a5a9-4ab0-8a7d-a70e1caa5885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.757437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0dced-a5a9-4ab0-8a7d-a70e1caa5885 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.768066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 518/1034] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:47.769112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0dced-a5a9-4ab0-8a7d-a70e1caa5885 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.835023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:47.839820 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b7623-408b-47ea-9d7c-286a5dd441f3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.875928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21609ac-9aa6-4a7e-8079-c0cf7362c769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.876726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21609ac-9aa6-4a7e-8079-c0cf7362c769 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.876834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0dced-a5a9-4ab0-8a7d-a70e1caa5885 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.877017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21609ac-9aa6-4a7e-8079-c0cf7362c769 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.877292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21609ac-9aa6-4a7e-8079-c0cf7362c769 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.884558 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21609ac-9aa6-4a7e-8079-c0cf7362c769 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.886865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21609ac-9aa6-4a7e-8079-c0cf7362c769 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f21609ac-9aa6-4a7e-8079-c0cf7362c769 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:47.902200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:47.903232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:47.904192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:47.904192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:47.904192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:47.904192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:47.904192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:47.904192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:47.904192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:47.904192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:47.904192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:47.904192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:47.904192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 517/1035] 10.222.0.42 () {56 vars in 1093 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:47.909199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21609ac-9aa6-4a7e-8079-c0cf7362c769 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.917914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c2efb-5c27-429a-8bd9-10c62da9e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.918589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c2efb-5c27-429a-8bd9-10c62da9e23a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.918872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c2efb-5c27-429a-8bd9-10c62da9e23a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.919149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c2efb-5c27-429a-8bd9-10c62da9e23a None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.920230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c2efb-5c27-429a-8bd9-10c62da9e23a None admin] RBAC: Adding query filter params (type=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:47.920540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c2efb-5c27-429a-8bd9-10c62da9e23a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.922584 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c2efb-5c27-429a-8bd9-10c62da9e23a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.927917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 519/1036] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:47.933750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c2efb-5c27-429a-8bd9-10c62da9e23a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.000523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:48.020881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21609ac-9aa6-4a7e-8079-c0cf7362c769 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.049656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c2efb-5c27-429a-8bd9-10c62da9e23a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.059666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e671bcb-6d0a-42e5-b996-9efbeff5d8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.060353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e671bcb-6d0a-42e5-b996-9efbeff5d8cb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.060638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e671bcb-6d0a-42e5-b996-9efbeff5d8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.060918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e671bcb-6d0a-42e5-b996-9efbeff5d8cb None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.061777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e671bcb-6d0a-42e5-b996-9efbeff5d8cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:48.062096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e671bcb-6d0a-42e5-b996-9efbeff5d8cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.063511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e671bcb-6d0a-42e5-b996-9efbeff5d8cb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:48.089102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 518/1037] 10.222.0.42 () {56 vars in 1133 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:48.090110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e671bcb-6d0a-42e5-b996-9efbeff5d8cb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.092003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6647d4-b000-454b-8d6d-da320efafd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.092689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6647d4-b000-454b-8d6d-da320efafd8b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.092981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6647d4-b000-454b-8d6d-da320efafd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.093262 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6647d4-b000-454b-8d6d-da320efafd8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.094128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6647d4-b000-454b-8d6d-da320efafd8b None admin] RBAC: Adding query filter params (interface=public, service_id=24e48dc1b04b48ed90343313b54ef9d2, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:48.094448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6647d4-b000-454b-8d6d-da320efafd8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.095875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6647d4-b000-454b-8d6d-da320efafd8b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:48.102382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 520/1038] 10.222.0.42 () {56 vars in 1251 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/endpoints?service_id=24e48dc1b04b48ed90343313b54ef9d2&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:48.107804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6647d4-b000-454b-8d6d-da320efafd8b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.113084 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:48.227345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e671bcb-6d0a-42e5-b996-9efbeff5d8cb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.233945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6647d4-b000-454b-8d6d-da320efafd8b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.259368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:48.266713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a29a57-d766-4337-aab4-f6e771ab6553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.267665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a29a57-d766-4337-aab4-f6e771ab6553 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.268018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a29a57-d766-4337-aab4-f6e771ab6553 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.268368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a29a57-d766-4337-aab4-f6e771ab6553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.272434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f69e4c-4b2e-4244-b895-712a6a4cd76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.273221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f69e4c-4b2e-4244-b895-712a6a4cd76d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.273504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f69e4c-4b2e-4244-b895-712a6a4cd76d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.273807 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f69e4c-4b2e-4244-b895-712a6a4cd76d None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.274494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f69e4c-4b2e-4244-b895-712a6a4cd76d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.276093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f69e4c-4b2e-4244-b895-712a6a4cd76d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:48.282659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 521/1039] 10.222.0.42 () {56 vars in 1087 bytes} [Wed May 25 16:07:48 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:07:48.358824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a29a57-d766-4337-aab4-f6e771ab6553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.360220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a29a57-d766-4337-aab4-f6e771ab6553 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:48.380360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:48.488079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 519/1040] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:48 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 16:07:48.494915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f69e4c-4b2e-4244-b895-712a6a4cd76d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.602745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f69e4c-4b2e-4244-b895-712a6a4cd76d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.638106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ae677-f512-40c5-82b6-ee05443c4781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.638710 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ae677-f512-40c5-82b6-ee05443c4781 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.638995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ae677-f512-40c5-82b6-ee05443c4781 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.639300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ae677-f512-40c5-82b6-ee05443c4781 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.651152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ae677-f512-40c5-82b6-ee05443c4781 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.652700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ae677-f512-40c5-82b6-ee05443c4781 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ba4ae677-f512-40c5-82b6-ee05443c4781 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:48.670630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:48.671628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:48.672547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:48.672547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:48.672547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:48.672547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:48.673398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 522/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:48.677708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a29a57-d766-4337-aab4-f6e771ab6553 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.785278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a29a57-d766-4337-aab4-f6e771ab6553 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.803309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:48.824774 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a7ccb-b8b3-4194-a3a8-eb1b5c919f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.825448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a7ccb-b8b3-4194-a3a8-eb1b5c919f43 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.825755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a7ccb-b8b3-4194-a3a8-eb1b5c919f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.826042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a7ccb-b8b3-4194-a3a8-eb1b5c919f43 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.826758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a7ccb-b8b3-4194-a3a8-eb1b5c919f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:48.827066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a7ccb-b8b3-4194-a3a8-eb1b5c919f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.828438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a7ccb-b8b3-4194-a3a8-eb1b5c919f43 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:48.845218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 520/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:48.846796 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ae677-f512-40c5-82b6-ee05443c4781 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.892603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:48.954841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ae677-f512-40c5-82b6-ee05443c4781 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.996094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff35cd-8b68-4231-81af-32f41b4d76c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.997266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff35cd-8b68-4231-81af-32f41b4d76c2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.997560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff35cd-8b68-4231-81af-32f41b4d76c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.998200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff35cd-8b68-4231-81af-32f41b4d76c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.009648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff35cd-8b68-4231-81af-32f41b4d76c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:49.011684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff35cd-8b68-4231-81af-32f41b4d76c2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-33ff35cd-8b68-4231-81af-32f41b4d76c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:49.028108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:49.029119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:49.030059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:49.030059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:49.030059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:49.030059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:49.030059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:49.030059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:49.030059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:49.030059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:49.031187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 523/1043] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:49.035713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a7ccb-b8b3-4194-a3a8-eb1b5c919f43 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.170784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a7ccb-b8b3-4194-a3a8-eb1b5c919f43 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.212341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f01c8-31e7-4d6c-a2fa-52e8bff230a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.213789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f01c8-31e7-4d6c-a2fa-52e8bff230a8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.214312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f01c8-31e7-4d6c-a2fa-52e8bff230a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.214592 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f01c8-31e7-4d6c-a2fa-52e8bff230a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.215373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f01c8-31e7-4d6c-a2fa-52e8bff230a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:49.215679 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f01c8-31e7-4d6c-a2fa-52e8bff230a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:49.217095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f01c8-31e7-4d6c-a2fa-52e8bff230a8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:49.229766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 521/1044] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:49.235757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57fcf3-a85b-41c5-ac04-b72f366a2e4a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.235757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57fcf3-a85b-41c5-ac04-b72f366a2e4a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.235757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57fcf3-a85b-41c5-ac04-b72f366a2e4a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.236062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f01c8-31e7-4d6c-a2fa-52e8bff230a8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.236745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 524/1045] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:49.241622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87c1fe-8504-4aab-a3c5-e3337961b308 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.241953 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87c1fe-8504-4aab-a3c5-e3337961b308 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.242229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87c1fe-8504-4aab-a3c5-e3337961b308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.261199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ec87c1fe-8504-4aab-a3c5-e3337961b308 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:49.262599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ec87c1fe-8504-4aab-a3c5-e3337961b308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:49.276076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:49.345907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f01c8-31e7-4d6c-a2fa-52e8bff230a8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.350013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 525/1046] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:49.361825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87c1fe-8504-4aab-a3c5-e3337961b308 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.382794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1190e9-2105-4c89-9fa1-ecf2e4f7f36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.383463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1190e9-2105-4c89-9fa1-ecf2e4f7f36c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.383749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1190e9-2105-4c89-9fa1-ecf2e4f7f36c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.384026 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1190e9-2105-4c89-9fa1-ecf2e4f7f36c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.385779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1190e9-2105-4c89-9fa1-ecf2e4f7f36c None admin] RBAC: Adding query filter params (role.id=73f22629793d47f086feb1e90de17023, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=6e6795b5b5cf4499ad494b875756ba71) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:49.385779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1190e9-2105-4c89-9fa1-ecf2e4f7f36c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:49.387147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1190e9-2105-4c89-9fa1-ecf2e4f7f36c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:49.401758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 522/1047] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:49 2022] GET /identity/v3/role_assignments?user.id=6e6795b5b5cf4499ad494b875756ba71&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=73f22629793d47f086feb1e90de17023 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:49.405751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:49.497950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87c1fe-8504-4aab-a3c5-e3337961b308 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.538252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eeecd0-1d4f-403b-8640-59402919617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.539076 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eeecd0-1d4f-403b-8640-59402919617f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.539358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eeecd0-1d4f-403b-8640-59402919617f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.539633 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eeecd0-1d4f-403b-8640-59402919617f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.551467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eeecd0-1d4f-403b-8640-59402919617f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e5eeecd0-1d4f-403b-8640-59402919617f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:49.560141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:49.561187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:49.562085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 526/1048] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:49.571087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1190e9-2105-4c89-9fa1-ecf2e4f7f36c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.673412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:49.677064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1190e9-2105-4c89-9fa1-ecf2e4f7f36c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.712881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38f295-e99f-44bd-a734-fe8433a839e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.713489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f295-e99f-44bd-a734-fe8433a839e8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.713793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f295-e99f-44bd-a734-fe8433a839e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.714071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f295-e99f-44bd-a734-fe8433a839e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.714843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38f295-e99f-44bd-a734-fe8433a839e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:49.715151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38f295-e99f-44bd-a734-fe8433a839e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:49.716515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38f295-e99f-44bd-a734-fe8433a839e8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:49.724812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 523/1049] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:07:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:49.731011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eeecd0-1d4f-403b-8640-59402919617f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.827176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:49.837665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eeecd0-1d4f-403b-8640-59402919617f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.878266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5933f8-9750-408b-9071-d41dd8f904a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.879143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5933f8-9750-408b-9071-d41dd8f904a2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.879435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5933f8-9750-408b-9071-d41dd8f904a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.879712 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5933f8-9750-408b-9071-d41dd8f904a2 None admin] PATH_INFO: `/v3/users/2e74334c6d4a45fe954ab944b2eb246b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.895452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5933f8-9750-408b-9071-d41dd8f904a2 None admin] RBAC: Authorizing `identity:get_user(user_id=2e74334c6d4a45fe954ab944b2eb246b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:49.896910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5933f8-9750-408b-9071-d41dd8f904a2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:49.898825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 527/1050] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:07:49 2022] GET /identity/v3/users/2e74334c6d4a45fe954ab944b2eb246b => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:49.904781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38f295-e99f-44bd-a734-fe8433a839e8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:50.009982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38f295-e99f-44bd-a734-fe8433a839e8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:50.044007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a1d59-652f-41f5-9678-257b3a71b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:50.044656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a1d59-652f-41f5-9678-257b3a71b6cc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:50.044941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a1d59-652f-41f5-9678-257b3a71b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:50.045354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a1d59-652f-41f5-9678-257b3a71b6cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:50.076550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:50.117891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a1d59-652f-41f5-9678-257b3a71b6cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:50.119402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a1d59-652f-41f5-9678-257b3a71b6cc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:50.225288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 524/1051] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:49 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:07:50.226744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5933f8-9750-408b-9071-d41dd8f904a2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:50.332971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5933f8-9750-408b-9071-d41dd8f904a2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:50.367618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fc66e-93d5-403f-a703-e5673252c25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:50.368469 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fc66e-93d5-403f-a703-e5673252c25b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:50.368752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fc66e-93d5-403f-a703-e5673252c25b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:50.369542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fc66e-93d5-403f-a703-e5673252c25b None admin] PATH_INFO: `/v3/projects/7536ba7fce1d4964aadf81b5304f145e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:50.372454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fc66e-93d5-403f-a703-e5673252c25b None admin] RBAC: Authorizing `identity:get_project(project_id=7536ba7fce1d4964aadf81b5304f145e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:50.373923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fc66e-93d5-403f-a703-e5673252c25b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:50.380546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 528/1052] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:07:50 2022] GET /identity/v3/projects/7536ba7fce1d4964aadf81b5304f145e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:50.381918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a1d59-652f-41f5-9678-257b3a71b6cc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:50.455026 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:50.496481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a1d59-652f-41f5-9678-257b3a71b6cc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:50.533766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a10a0-ba95-40e2-958f-6d186cafcc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:50.534433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a10a0-ba95-40e2-958f-6d186cafcc9b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:50.534707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a10a0-ba95-40e2-958f-6d186cafcc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:50.534982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a10a0-ba95-40e2-958f-6d186cafcc9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:50.535801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a10a0-ba95-40e2-958f-6d186cafcc9b None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=7536ba7fce1d4964aadf81b5304f145e, user.id=2e74334c6d4a45fe954ab944b2eb246b) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:50.536111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a10a0-ba95-40e2-958f-6d186cafcc9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:50.537525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a10a0-ba95-40e2-958f-6d186cafcc9b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:50.549759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 525/1053] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:50 2022] GET /identity/v3/role_assignments?user.id=2e74334c6d4a45fe954ab944b2eb246b&scope.project.id=7536ba7fce1d4964aadf81b5304f145e&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:50.593967 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91a8b5-1842-4554-bd93-00bf4a8752e8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:50.594512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91a8b5-1842-4554-bd93-00bf4a8752e8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:50.594512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91a8b5-1842-4554-bd93-00bf4a8752e8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:50.595915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 529/1054] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:50.601942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283f1ba-1e39-46b1-bcd1-64b981746863 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:50.602551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283f1ba-1e39-46b1-bcd1-64b981746863 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:50.602551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283f1ba-1e39-46b1-bcd1-64b981746863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:50.622099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5283f1ba-1e39-46b1-bcd1-64b981746863 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:50.623322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5283f1ba-1e39-46b1-bcd1-64b981746863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:50.701924 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 526/1055] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:50.725624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91a8b5-1842-4554-bd93-00bf4a8752e8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:50.745223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:50.832378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91a8b5-1842-4554-bd93-00bf4a8752e8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:50.868050 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a3fc4-b60c-4aef-b7f6-26dbb035e3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:50.868764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a3fc4-b60c-4aef-b7f6-26dbb035e3ce None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:50.869053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a3fc4-b60c-4aef-b7f6-26dbb035e3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:50.869333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a3fc4-b60c-4aef-b7f6-26dbb035e3ce None admin] PATH_INFO: `/v3/services/image` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:50.870023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a3fc4-b60c-4aef-b7f6-26dbb035e3ce None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:50.871429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a3fc4-b60c-4aef-b7f6-26dbb035e3ce None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-510a3fc4-b60c-4aef-b7f6-26dbb035e3ce None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:50.882968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:50.884040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:50.885860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 530/1056] 10.222.0.42 () {56 vars in 1105 bytes} [Wed May 25 16:07:50 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:50.890760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283f1ba-1e39-46b1-bcd1-64b981746863 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:50.952254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:50.997983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283f1ba-1e39-46b1-bcd1-64b981746863 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:51.034362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ccf21-3bf2-47c9-88bf-c73cbf9e92a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:51.035042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ccf21-3bf2-47c9-88bf-c73cbf9e92a0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:51.035326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ccf21-3bf2-47c9-88bf-c73cbf9e92a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:51.035601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ccf21-3bf2-47c9-88bf-c73cbf9e92a0 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:51.036309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ccf21-3bf2-47c9-88bf-c73cbf9e92a0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:51.036616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ccf21-3bf2-47c9-88bf-c73cbf9e92a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:51.038004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ccf21-3bf2-47c9-88bf-c73cbf9e92a0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:51.047571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 527/1057] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:51.049086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a3fc4-b60c-4aef-b7f6-26dbb035e3ce None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:51.137230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:51.158324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a3fc4-b60c-4aef-b7f6-26dbb035e3ce None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:51.192966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ee30b-ac6d-464a-bac5-045d8dce4d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:51.193645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ee30b-ac6d-464a-bac5-045d8dce4d71 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:51.193956 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ee30b-ac6d-464a-bac5-045d8dce4d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:51.194237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ee30b-ac6d-464a-bac5-045d8dce4d71 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:51.194944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ee30b-ac6d-464a-bac5-045d8dce4d71 None admin] RBAC: Adding query filter params (type=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:51.195251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ee30b-ac6d-464a-bac5-045d8dce4d71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:51.196628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ee30b-ac6d-464a-bac5-045d8dce4d71 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:51.201994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 531/1058] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:51.208344 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ccf21-3bf2-47c9-88bf-c73cbf9e92a0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:51.312130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ccf21-3bf2-47c9-88bf-c73cbf9e92a0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:51.332322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:51.348363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2faf1f-ffd1-4c53-84e9-6e8b54889034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:51.349033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2faf1f-ffd1-4c53-84e9-6e8b54889034 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:51.349317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2faf1f-ffd1-4c53-84e9-6e8b54889034 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:51.349595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2faf1f-ffd1-4c53-84e9-6e8b54889034 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:51.350375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2faf1f-ffd1-4c53-84e9-6e8b54889034 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:51.351753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2faf1f-ffd1-4c53-84e9-6e8b54889034 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:51.373045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 528/1059] 10.222.0.42 () {60 vars in 1144 bytes} [Wed May 25 16:07:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:07:51.652104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053554e-01f4-4817-bf6f-acc60e56568b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:51.652665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053554e-01f4-4817-bf6f-acc60e56568b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:51.652665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053554e-01f4-4817-bf6f-acc60e56568b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:51.654232 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 532/1060] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:51.660054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326572d-f7c7-4b77-b785-99b8cfd60817 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:51.660459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326572d-f7c7-4b77-b785-99b8cfd60817 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:51.660626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326572d-f7c7-4b77-b785-99b8cfd60817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:51.678309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3326572d-f7c7-4b77-b785-99b8cfd60817 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:51.679586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3326572d-f7c7-4b77-b785-99b8cfd60817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:51.758843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 529/1061] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:51.781867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053554e-01f4-4817-bf6f-acc60e56568b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:51.886913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053554e-01f4-4817-bf6f-acc60e56568b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:51.925237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc076d-6134-4194-83d8-995502342f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:51.925936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc076d-6134-4194-83d8-995502342f98 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:51.926221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc076d-6134-4194-83d8-995502342f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:51.926496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc076d-6134-4194-83d8-995502342f98 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:51.937693 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc076d-6134-4194-83d8-995502342f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5ebc076d-6134-4194-83d8-995502342f98 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:51.945950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:51.946972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:51.948245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 533/1062] 10.222.0.42 () {56 vars in 1120 bytes} [Wed May 25 16:07:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:51.953287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326572d-f7c7-4b77-b785-99b8cfd60817 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:52.056857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326572d-f7c7-4b77-b785-99b8cfd60817 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:52.058665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:52.094719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf75a2-0cc5-4eab-bb7c-35aed796bd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:52.095404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf75a2-0cc5-4eab-bb7c-35aed796bd2e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:52.095692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf75a2-0cc5-4eab-bb7c-35aed796bd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:52.095971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf75a2-0cc5-4eab-bb7c-35aed796bd2e None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:52.096782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf75a2-0cc5-4eab-bb7c-35aed796bd2e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:52.097093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf75a2-0cc5-4eab-bb7c-35aed796bd2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:52.098487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf75a2-0cc5-4eab-bb7c-35aed796bd2e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:52.111976 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 530/1063] 10.222.0.42 () {56 vars in 1115 bytes} [Wed May 25 16:07:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:52.113453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc076d-6134-4194-83d8-995502342f98 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:52.117309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:52.219420 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc076d-6134-4194-83d8-995502342f98 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:52.253646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892ac7a-2422-4b35-a213-58eed056bcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:52.254813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892ac7a-2422-4b35-a213-58eed056bcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:52.255257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892ac7a-2422-4b35-a213-58eed056bcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:52.255697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892ac7a-2422-4b35-a213-58eed056bcb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:52.267472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892ac7a-2422-4b35-a213-58eed056bcb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:52.269163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892ac7a-2422-4b35-a213-58eed056bcb6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7892ac7a-2422-4b35-a213-58eed056bcb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:52.286950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:52.288106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:52.289325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:52.289325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:52.289325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:52.289325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:52.291943 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 534/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:52.297355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf75a2-0cc5-4eab-bb7c-35aed796bd2e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:52.400557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf75a2-0cc5-4eab-bb7c-35aed796bd2e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:52.444200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858f5f1-17c7-4a16-92b4-d8db3a608345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:52.444880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f5f1-17c7-4a16-92b4-d8db3a608345 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:52.445164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f5f1-17c7-4a16-92b4-d8db3a608345 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:52.445442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f5f1-17c7-4a16-92b4-d8db3a608345 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:52.446198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858f5f1-17c7-4a16-92b4-d8db3a608345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:52.446506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858f5f1-17c7-4a16-92b4-d8db3a608345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:52.448022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858f5f1-17c7-4a16-92b4-d8db3a608345 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:52.464943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 531/1065] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:52.466382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892ac7a-2422-4b35-a213-58eed056bcb6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:52.524179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:52.573798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892ac7a-2422-4b35-a213-58eed056bcb6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:52.609563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e7cf2-7c9d-48f2-bf7e-a77729f47651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:52.610535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e7cf2-7c9d-48f2-bf7e-a77729f47651 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:52.610826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e7cf2-7c9d-48f2-bf7e-a77729f47651 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:52.611108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e7cf2-7c9d-48f2-bf7e-a77729f47651 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:52.620161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e7cf2-7c9d-48f2-bf7e-a77729f47651 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:52.621663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e7cf2-7c9d-48f2-bf7e-a77729f47651 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8a5e7cf2-7c9d-48f2-bf7e-a77729f47651 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:52.637919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:52.638972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:52.645296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:52.645296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:52.645296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:52.645296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:52.645296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:52.645296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:52.645296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:52.645296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:52.645296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:52.645296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:52.645296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 535/1066] 10.222.0.42 () {56 vars in 1093 bytes} [Wed May 25 16:07:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:52.645971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858f5f1-17c7-4a16-92b4-d8db3a608345 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:52.672701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:52.750987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858f5f1-17c7-4a16-92b4-d8db3a608345 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:52.786459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ccad4-7531-4854-8d5e-4a03a4ab36c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:52.787132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ccad4-7531-4854-8d5e-4a03a4ab36c5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:52.787419 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ccad4-7531-4854-8d5e-4a03a4ab36c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:52.787698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ccad4-7531-4854-8d5e-4a03a4ab36c5 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:52.788528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ccad4-7531-4854-8d5e-4a03a4ab36c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:52.788840 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ccad4-7531-4854-8d5e-4a03a4ab36c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:52.790300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ccad4-7531-4854-8d5e-4a03a4ab36c5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:52.812798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 532/1067] 10.222.0.42 () {56 vars in 1133 bytes} [Wed May 25 16:07:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:52.814426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e7cf2-7c9d-48f2-bf7e-a77729f47651 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:52.823337 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9650b-e608-41e5-8873-e65bad37b1b1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:52.823627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9650b-e608-41e5-8873-e65bad37b1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:52.823909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9650b-e608-41e5-8873-e65bad37b1b1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:52.825279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 533/1068] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:07:52.830381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1fa81-382e-4b1b-8649-63c4f1eb2f76 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:52.830672 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1fa81-382e-4b1b-8649-63c4f1eb2f76 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:52.830951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1fa81-382e-4b1b-8649-63c4f1eb2f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:52.848461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9cf1fa81-382e-4b1b-8649-63c4f1eb2f76 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:52.849648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9cf1fa81-382e-4b1b-8649-63c4f1eb2f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:52.918245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 534/1069] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:52.935498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e7cf2-7c9d-48f2-bf7e-a77729f47651 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:52.941295 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1fa81-382e-4b1b-8649-63c4f1eb2f76 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:52.955748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:52.955920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:52.977458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fdd74-6622-4b1c-af37-74994fea82ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:52.978446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fdd74-6622-4b1c-af37-74994fea82ee None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:52.978766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fdd74-6622-4b1c-af37-74994fea82ee None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:52.979678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fdd74-6622-4b1c-af37-74994fea82ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:52.991315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fdd74-6622-4b1c-af37-74994fea82ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:52.992770 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fdd74-6622-4b1c-af37-74994fea82ee None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-cb8fdd74-6622-4b1c-af37-74994fea82ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:53.011187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:53.012150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:53.013064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:53.013064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:53.013064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:53.013064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:53.013064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 536/1070] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:53.017954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fdd74-6622-4b1c-af37-74994fea82ee None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:53.058245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1fa81-382e-4b1b-8649-63c4f1eb2f76 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:53.102068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa556d-9be1-46f3-b773-e9cb579657cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:53.103416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa556d-9be1-46f3-b773-e9cb579657cb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.103805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa556d-9be1-46f3-b773-e9cb579657cb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.104378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa556d-9be1-46f3-b773-e9cb579657cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.119271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa556d-9be1-46f3-b773-e9cb579657cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:53.126741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fdd74-6622-4b1c-af37-74994fea82ee None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4afa556d-9be1-46f3-b773-e9cb579657cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:53.132189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:53.133167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:53.134045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 535/1071] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:53.140133 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa556d-9be1-46f3-b773-e9cb579657cb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:53.169081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0fcfb-b520-4bd5-936e-0f09638386c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:53.169771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0fcfb-b520-4bd5-936e-0f09638386c6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.170065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0fcfb-b520-4bd5-936e-0f09638386c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.170338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0fcfb-b520-4bd5-936e-0f09638386c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.171061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0fcfb-b520-4bd5-936e-0f09638386c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:53.171367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0fcfb-b520-4bd5-936e-0f09638386c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:53.172736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0fcfb-b520-4bd5-936e-0f09638386c6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:53.186751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 537/1072] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:53.192144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0fcfb-b520-4bd5-936e-0f09638386c6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:53.222172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:53.222818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:53.256174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa556d-9be1-46f3-b773-e9cb579657cb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:53.293064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fae08-c31b-4c5a-8277-beaf00985604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:53.293771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fae08-c31b-4c5a-8277-beaf00985604 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.294063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fae08-c31b-4c5a-8277-beaf00985604 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.294342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fae08-c31b-4c5a-8277-beaf00985604 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.295100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fae08-c31b-4c5a-8277-beaf00985604 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:53.295402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fae08-c31b-4c5a-8277-beaf00985604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:53.296781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fae08-c31b-4c5a-8277-beaf00985604 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:53.305609 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0fcfb-b520-4bd5-936e-0f09638386c6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:53.311298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 536/1073] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:07:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:53.312351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fae08-c31b-4c5a-8277-beaf00985604 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:53.352540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840db5b-bb71-4898-8833-e37f3e8f7ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:53.353758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840db5b-bb71-4898-8833-e37f3e8f7ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.354038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840db5b-bb71-4898-8833-e37f3e8f7ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.354512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840db5b-bb71-4898-8833-e37f3e8f7ae3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.364855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840db5b-bb71-4898-8833-e37f3e8f7ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:53.366746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840db5b-bb71-4898-8833-e37f3e8f7ae3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d840db5b-bb71-4898-8833-e37f3e8f7ae3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:53.385306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:53.386804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:53.388013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:53.388013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:53.388013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:53.388013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:53.388013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:53.388013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:53.388013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:53.388013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:53.391629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 538/1074] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:53.392406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840db5b-bb71-4898-8833-e37f3e8f7ae3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:53.424689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fae08-c31b-4c5a-8277-beaf00985604 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:53.463897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1dee4-881d-40fd-b058-88681f07cbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:53.464480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1dee4-881d-40fd-b058-88681f07cbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.464767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1dee4-881d-40fd-b058-88681f07cbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.465088 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1dee4-881d-40fd-b058-88681f07cbb4 None admin] PATH_INFO: `/v3/users/2e74334c6d4a45fe954ab944b2eb246b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.466402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1dee4-881d-40fd-b058-88681f07cbb4 None admin] RBAC: Authorizing `identity:get_user(user_id=2e74334c6d4a45fe954ab944b2eb246b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:53.467944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1dee4-881d-40fd-b058-88681f07cbb4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:53.469761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 537/1075] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:07:53 2022] GET /identity/v3/users/2e74334c6d4a45fe954ab944b2eb246b => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:53.475857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1dee4-881d-40fd-b058-88681f07cbb4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:53.479375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:53.501574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:53.516016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840db5b-bb71-4898-8833-e37f3e8f7ae3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:53.557381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536feb-7cfd-4cbd-bc74-40729fd241b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:53.558092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536feb-7cfd-4cbd-bc74-40729fd241b7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.558372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536feb-7cfd-4cbd-bc74-40729fd241b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.558646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536feb-7cfd-4cbd-bc74-40729fd241b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.559451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59536feb-7cfd-4cbd-bc74-40729fd241b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:53.559763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59536feb-7cfd-4cbd-bc74-40729fd241b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:53.561986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59536feb-7cfd-4cbd-bc74-40729fd241b7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:53.581769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 539/1076] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:53.584917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536feb-7cfd-4cbd-bc74-40729fd241b7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:53.592296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1dee4-881d-40fd-b058-88681f07cbb4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:53.629929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27229453-3686-4488-8def-2590026f334c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:53.630798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27229453-3686-4488-8def-2590026f334c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.631078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27229453-3686-4488-8def-2590026f334c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.631348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27229453-3686-4488-8def-2590026f334c None admin] PATH_INFO: `/v3/projects/7536ba7fce1d4964aadf81b5304f145e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.632565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27229453-3686-4488-8def-2590026f334c None admin] RBAC: Authorizing `identity:get_project(project_id=7536ba7fce1d4964aadf81b5304f145e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:53.634002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27229453-3686-4488-8def-2590026f334c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:53.635992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 538/1077] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:07:53 2022] GET /identity/v3/projects/7536ba7fce1d4964aadf81b5304f145e => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:53.641539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27229453-3686-4488-8def-2590026f334c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:53.694920 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536feb-7cfd-4cbd-bc74-40729fd241b7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:53.713498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:53.729161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f719ed-e007-43c1-ac51-80f55233b8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:53.730003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f719ed-e007-43c1-ac51-80f55233b8f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.730280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f719ed-e007-43c1-ac51-80f55233b8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.730548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f719ed-e007-43c1-ac51-80f55233b8f1 None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/6e6795b5b5cf4499ad494b875756ba71/roles/73f22629793d47f086feb1e90de17023` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.732756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f719ed-e007-43c1-ac51-80f55233b8f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=6e6795b5b5cf4499ad494b875756ba71, role_id=73f22629793d47f086feb1e90de17023)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:53.734228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f719ed-e007-43c1-ac51-80f55233b8f1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:53.747765 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 540/1078] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:07:53 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/6e6795b5b5cf4499ad494b875756ba71/roles/73f22629793d47f086feb1e90de17023 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:07:53.747924 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27229453-3686-4488-8def-2590026f334c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:53.749022 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dca229-0497-4194-a877-1d57c15d9b3b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.749304 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dca229-0497-4194-a877-1d57c15d9b3b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.749578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dca229-0497-4194-a877-1d57c15d9b3b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.750895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 541/1079] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:53.767856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ca645-dafe-42db-9b1b-40cf72240feb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.768170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ca645-dafe-42db-9b1b-40cf72240feb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.768463 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ca645-dafe-42db-9b1b-40cf72240feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.790642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-486ca645-dafe-42db-9b1b-40cf72240feb None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:53.792466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-486ca645-dafe-42db-9b1b-40cf72240feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:53.795027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:53.798279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee3139-b367-4046-bb2a-264c34011400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:53.799301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee3139-b367-4046-bb2a-264c34011400 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.799587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee3139-b367-4046-bb2a-264c34011400 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.799865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee3139-b367-4046-bb2a-264c34011400 None admin] PATH_INFO: `/v3/projects/7536ba7fce1d4964aadf81b5304f145e/users/2e74334c6d4a45fe954ab944b2eb246b/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.802005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee3139-b367-4046-bb2a-264c34011400 None admin] RBAC: Authorizing `identity:create_grant(project_id=7536ba7fce1d4964aadf81b5304f145e, user_id=2e74334c6d4a45fe954ab944b2eb246b, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:53.803462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee3139-b367-4046-bb2a-264c34011400 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:53.821214 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 539/1080] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:07:53 2022] PUT /identity/v3/projects/7536ba7fce1d4964aadf81b5304f145e/users/2e74334c6d4a45fe954ab944b2eb246b/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:07:53.885750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 542/1081] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:53.904878 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee3139-b367-4046-bb2a-264c34011400 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:54.029042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee3139-b367-4046-bb2a-264c34011400 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:54.058994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:54.067921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126678b-e2e9-43dc-8cc1-4358bdeaceb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:54.068581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126678b-e2e9-43dc-8cc1-4358bdeaceb5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:54.068860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126678b-e2e9-43dc-8cc1-4358bdeaceb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:54.069133 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126678b-e2e9-43dc-8cc1-4358bdeaceb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:54.161982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126678b-e2e9-43dc-8cc1-4358bdeaceb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:54.163490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126678b-e2e9-43dc-8cc1-4358bdeaceb5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:54.281293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 540/1082] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 16:07:54.282505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ca645-dafe-42db-9b1b-40cf72240feb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:54.386362 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ca645-dafe-42db-9b1b-40cf72240feb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:54.423067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a6e25-6efe-4f25-b58c-01db2d14efa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:54.423863 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a6e25-6efe-4f25-b58c-01db2d14efa6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:54.424147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a6e25-6efe-4f25-b58c-01db2d14efa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:54.424418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a6e25-6efe-4f25-b58c-01db2d14efa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:54.433011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:54.438408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a6e25-6efe-4f25-b58c-01db2d14efa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:54.440603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a6e25-6efe-4f25-b58c-01db2d14efa6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f69a6e25-6efe-4f25-b58c-01db2d14efa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:54.458639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:54.459599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:54.460543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:54.460543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:54.460543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:54.460543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:54.461384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 543/1083] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:54.466801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126678b-e2e9-43dc-8cc1-4358bdeaceb5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:54.504623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:54.582730 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126678b-e2e9-43dc-8cc1-4358bdeaceb5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:54.620841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc1aca-0a1e-47b6-9f2e-6207acb38de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:54.621520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc1aca-0a1e-47b6-9f2e-6207acb38de9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:54.621829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc1aca-0a1e-47b6-9f2e-6207acb38de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:54.622107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc1aca-0a1e-47b6-9f2e-6207acb38de9 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:54.622836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc1aca-0a1e-47b6-9f2e-6207acb38de9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:54.623139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc1aca-0a1e-47b6-9f2e-6207acb38de9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:54.624689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc1aca-0a1e-47b6-9f2e-6207acb38de9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:54.645749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 541/1084] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:56.022094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9fa4f-a2ed-4c80-8b98-f7b6fed63e36 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.022629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9fa4f-a2ed-4c80-8b98-f7b6fed63e36 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.022629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9fa4f-a2ed-4c80-8b98-f7b6fed63e36 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.024039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 544/1085] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:56.029755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615ccb-2c2f-4cdc-941c-07abf2200bad None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.030178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615ccb-2c2f-4cdc-941c-07abf2200bad None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.030321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615ccb-2c2f-4cdc-941c-07abf2200bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.047573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8b615ccb-2c2f-4cdc-941c-07abf2200bad None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:56.048746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8b615ccb-2c2f-4cdc-941c-07abf2200bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:56.113846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30796a2f-7f00-436f-9dca-0006c3dc589a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.114142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30796a2f-7f00-436f-9dca-0006c3dc589a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.114419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30796a2f-7f00-436f-9dca-0006c3dc589a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.115800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 545/1086] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:56.119975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 542/1087] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:56.122153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c0af-bea2-4b94-9a22-80ebf799d11c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.122515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c0af-bea2-4b94-9a22-80ebf799d11c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.122855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c0af-bea2-4b94-9a22-80ebf799d11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.140098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d337c0af-bea2-4b94-9a22-80ebf799d11c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:56.141372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d337c0af-bea2-4b94-9a22-80ebf799d11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:56.144239 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b615ccb-2c2f-4cdc-941c-07abf2200bad None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:56.167321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:56.211045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 546/1088] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:07:56.234669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337c0af-bea2-4b94-9a22-80ebf799d11c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:56.248206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b615ccb-2c2f-4cdc-941c-07abf2200bad None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:56.283888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703b857-226b-46a5-80f3-c8e2a0cb720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:56.284629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703b857-226b-46a5-80f3-c8e2a0cb720d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.284910 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703b857-226b-46a5-80f3-c8e2a0cb720d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.285185 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703b857-226b-46a5-80f3-c8e2a0cb720d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.296153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703b857-226b-46a5-80f3-c8e2a0cb720d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9703b857-226b-46a5-80f3-c8e2a0cb720d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:56.304256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:56.305341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:56.306368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 543/1089] 10.222.0.42 () {56 vars in 1120 bytes} [Wed May 25 16:07:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:56.311452 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703b857-226b-46a5-80f3-c8e2a0cb720d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:56.334356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337c0af-bea2-4b94-9a22-80ebf799d11c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:56.355204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:56.370441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3505fe-9ece-45ac-a092-e934667357b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:56.371238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3505fe-9ece-45ac-a092-e934667357b9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.371524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3505fe-9ece-45ac-a092-e934667357b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.371800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3505fe-9ece-45ac-a092-e934667357b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.381823 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3505fe-9ece-45ac-a092-e934667357b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2f3505fe-9ece-45ac-a092-e934667357b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:07:56.389538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:56.390846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:56.392484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 547/1090] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:07:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:56.397530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3505fe-9ece-45ac-a092-e934667357b9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:56.399759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:56.424408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703b857-226b-46a5-80f3-c8e2a0cb720d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:56.458779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec3d24-8b19-41cd-a2ab-18087b8be242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:56.459450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec3d24-8b19-41cd-a2ab-18087b8be242 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.459814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec3d24-8b19-41cd-a2ab-18087b8be242 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.460015 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec3d24-8b19-41cd-a2ab-18087b8be242 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.460792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec3d24-8b19-41cd-a2ab-18087b8be242 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:56.461105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec3d24-8b19-41cd-a2ab-18087b8be242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:56.463014 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec3d24-8b19-41cd-a2ab-18087b8be242 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:56.470929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 544/1091] 10.222.0.42 () {56 vars in 1115 bytes} [Wed May 25 16:07:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:56.476551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec3d24-8b19-41cd-a2ab-18087b8be242 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:56.506109 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3505fe-9ece-45ac-a092-e934667357b9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:56.539774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a8fcd-2d71-4284-b51b-662bd1ca198e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:56.540362 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8fcd-2d71-4284-b51b-662bd1ca198e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.540700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8fcd-2d71-4284-b51b-662bd1ca198e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.540983 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8fcd-2d71-4284-b51b-662bd1ca198e None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.541754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a8fcd-2d71-4284-b51b-662bd1ca198e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:56.542063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a8fcd-2d71-4284-b51b-662bd1ca198e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:56.543432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a8fcd-2d71-4284-b51b-662bd1ca198e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:56.551428 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 548/1092] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:07:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:56.558204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a8fcd-2d71-4284-b51b-662bd1ca198e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:56.578703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec3d24-8b19-41cd-a2ab-18087b8be242 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:56.601965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:56.613398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e83c6d-3a0c-494a-bf4c-18ddf6107d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:56.614233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e83c6d-3a0c-494a-bf4c-18ddf6107d7c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.614509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e83c6d-3a0c-494a-bf4c-18ddf6107d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.614782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e83c6d-3a0c-494a-bf4c-18ddf6107d7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.625892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e83c6d-3a0c-494a-bf4c-18ddf6107d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:56.627331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e83c6d-3a0c-494a-bf4c-18ddf6107d7c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:56.629120 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d5e83c6d-3a0c-494a-bf4c-18ddf6107d7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:56.645661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:56.646642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:56.647539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:56.647539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:56.647539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:56.647539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:56.647539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 545/1093] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:56.652628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e83c6d-3a0c-494a-bf4c-18ddf6107d7c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:56.662504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a8fcd-2d71-4284-b51b-662bd1ca198e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:56.696621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad151c75-1bba-4b34-a401-9ede06221567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:56.697938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad151c75-1bba-4b34-a401-9ede06221567 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.698221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad151c75-1bba-4b34-a401-9ede06221567 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.698493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad151c75-1bba-4b34-a401-9ede06221567 None admin] PATH_INFO: `/v3/users/2e74334c6d4a45fe954ab944b2eb246b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.699690 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad151c75-1bba-4b34-a401-9ede06221567 None admin] RBAC: Authorizing `identity:get_user(user_id=2e74334c6d4a45fe954ab944b2eb246b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:56.701112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad151c75-1bba-4b34-a401-9ede06221567 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:56.702925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 549/1094] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:07:56 2022] GET /identity/v3/users/2e74334c6d4a45fe954ab944b2eb246b => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:56.708351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad151c75-1bba-4b34-a401-9ede06221567 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:56.754528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e83c6d-3a0c-494a-bf4c-18ddf6107d7c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:56.790023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bef1a3-51a3-4d70-984b-d9e780e72fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:56.791123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bef1a3-51a3-4d70-984b-d9e780e72fdd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.791411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bef1a3-51a3-4d70-984b-d9e780e72fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.791682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bef1a3-51a3-4d70-984b-d9e780e72fdd None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.792569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bef1a3-51a3-4d70-984b-d9e780e72fdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:56.792879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bef1a3-51a3-4d70-984b-d9e780e72fdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:56.794451 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bef1a3-51a3-4d70-984b-d9e780e72fdd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:56.806362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 546/1095] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:56.811955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bef1a3-51a3-4d70-984b-d9e780e72fdd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:56.812503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad151c75-1bba-4b34-a401-9ede06221567 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:56.823564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:56.848540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776003-9da6-4575-9fd5-bd7540be5d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:56.849185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776003-9da6-4575-9fd5-bd7540be5d4f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.849518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776003-9da6-4575-9fd5-bd7540be5d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.849768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776003-9da6-4575-9fd5-bd7540be5d4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.868305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:56.915064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bef1a3-51a3-4d70-984b-d9e780e72fdd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:56.919202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776003-9da6-4575-9fd5-bd7540be5d4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:56.920664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776003-9da6-4575-9fd5-bd7540be5d4f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:56.949415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f171c-9623-49ab-91c4-ed423ddc1907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:56.950241 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f171c-9623-49ab-91c4-ed423ddc1907 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.950760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f171c-9623-49ab-91c4-ed423ddc1907 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.950873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f171c-9623-49ab-91c4-ed423ddc1907 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.957920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f171c-9623-49ab-91c4-ed423ddc1907 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:56.959356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f171c-9623-49ab-91c4-ed423ddc1907 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-839f171c-9623-49ab-91c4-ed423ddc1907 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:07:56.973900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:56.975053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:56.976367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:56.976367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:56.976367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:07:56.976367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:07:56.976367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:07:56.976367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:07:56.976367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:07:56.976367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:07:56.976367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 16:07:56.976367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:07:56.976367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 547/1096] 10.222.0.42 () {56 vars in 1093 bytes} [Wed May 25 16:07:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:56.981415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f171c-9623-49ab-91c4-ed423ddc1907 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:57.024808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 550/1097] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 16:07:57.031060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776003-9da6-4575-9fd5-bd7540be5d4f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:57.043130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:57.084530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f171c-9623-49ab-91c4-ed423ddc1907 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:57.104642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:57.129330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49368a26-3437-4edd-b4e4-058ad2decf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:57.130381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368a26-3437-4edd-b4e4-058ad2decf68 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:57.130809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368a26-3437-4edd-b4e4-058ad2decf68 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:57.131233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368a26-3437-4edd-b4e4-058ad2decf68 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:57.132574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49368a26-3437-4edd-b4e4-058ad2decf68 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:57.133058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49368a26-3437-4edd-b4e4-058ad2decf68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:57.134933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49368a26-3437-4edd-b4e4-058ad2decf68 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:57.141771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776003-9da6-4575-9fd5-bd7540be5d4f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:57.154593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 548/1098] 10.222.0.42 () {56 vars in 1133 bytes} [Wed May 25 16:07:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:57.160498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49368a26-3437-4edd-b4e4-058ad2decf68 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:57.178984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1af1e-e58e-45fb-8eb0-da660bc37286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:57.179869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1af1e-e58e-45fb-8eb0-da660bc37286 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:57.180152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1af1e-e58e-45fb-8eb0-da660bc37286 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:57.180426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1af1e-e58e-45fb-8eb0-da660bc37286 None admin] PATH_INFO: `/v3/projects/7536ba7fce1d4964aadf81b5304f145e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:57.181830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1af1e-e58e-45fb-8eb0-da660bc37286 None admin] RBAC: Authorizing `identity:get_project(project_id=7536ba7fce1d4964aadf81b5304f145e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:57.183679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1af1e-e58e-45fb-8eb0-da660bc37286 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:57.185453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 551/1099] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:07:57 2022] GET /identity/v3/projects/7536ba7fce1d4964aadf81b5304f145e => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:57.192197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1af1e-e58e-45fb-8eb0-da660bc37286 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:57.263929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49368a26-3437-4edd-b4e4-058ad2decf68 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:57.269091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:57.291746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1af1e-e58e-45fb-8eb0-da660bc37286 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:57.298970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39ec24-225e-42f7-8f11-24100a66d9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:57.299540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39ec24-225e-42f7-8f11-24100a66d9ef None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:57.299820 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39ec24-225e-42f7-8f11-24100a66d9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:57.300090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39ec24-225e-42f7-8f11-24100a66d9ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:57.327007 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41c22c-8f2c-477d-bb31-93432b638eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:57.327653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41c22c-8f2c-477d-bb31-93432b638eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:57.327992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41c22c-8f2c-477d-bb31-93432b638eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:57.328221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41c22c-8f2c-477d-bb31-93432b638eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:57.329041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41c22c-8f2c-477d-bb31-93432b638eb2 None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=7536ba7fce1d4964aadf81b5304f145e, user.id=2e74334c6d4a45fe954ab944b2eb246b) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:57.329354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41c22c-8f2c-477d-bb31-93432b638eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:57.330812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41c22c-8f2c-477d-bb31-93432b638eb2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:57.337960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 552/1100] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:57 2022] GET /identity/v3/role_assignments?user.id=2e74334c6d4a45fe954ab944b2eb246b&scope.project.id=7536ba7fce1d4964aadf81b5304f145e&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:07:57.342244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:57.372342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39ec24-225e-42f7-8f11-24100a66d9ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:57.373853 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39ec24-225e-42f7-8f11-24100a66d9ef None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:57.486735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 549/1101] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:07:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 16:07:57.492841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41c22c-8f2c-477d-bb31-93432b638eb2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:57.599561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41c22c-8f2c-477d-bb31-93432b638eb2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:57.638912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd92c-f380-449e-ad4b-a17c7e757a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:57.639717 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd92c-f380-449e-ad4b-a17c7e757a28 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:57.639996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd92c-f380-449e-ad4b-a17c7e757a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:57.640279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd92c-f380-449e-ad4b-a17c7e757a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:57.653445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abd92c-f380-449e-ad4b-a17c7e757a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:57.654962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abd92c-f380-449e-ad4b-a17c7e757a28 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:57.656927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-13abd92c-f380-449e-ad4b-a17c7e757a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:07:57.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:57.675040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:57.676125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:07:57.676125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:07:57.676125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:07:57.676125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:57.676599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 553/1102] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:07:57.681041 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39ec24-225e-42f7-8f11-24100a66d9ef None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:57.752895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:57.793294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39ec24-225e-42f7-8f11-24100a66d9ef None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:57.830368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dc1e6-ced5-4449-82cd-9d320330de34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:57.831118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dc1e6-ced5-4449-82cd-9d320330de34 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:57.831468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dc1e6-ced5-4449-82cd-9d320330de34 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:57.831808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dc1e6-ced5-4449-82cd-9d320330de34 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:57.832599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dc1e6-ced5-4449-82cd-9d320330de34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:57.832971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dc1e6-ced5-4449-82cd-9d320330de34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:57.834464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dc1e6-ced5-4449-82cd-9d320330de34 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:57.846407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 550/1103] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:57.852023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd92c-f380-449e-ad4b-a17c7e757a28 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:57.950726 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd92c-f380-449e-ad4b-a17c7e757a28 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:57.983186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf88ab-cb81-4ebe-8a79-2cd2fecd9f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:57.983979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf88ab-cb81-4ebe-8a79-2cd2fecd9f33 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:57.984260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf88ab-cb81-4ebe-8a79-2cd2fecd9f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:57.984531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf88ab-cb81-4ebe-8a79-2cd2fecd9f33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:57.991004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf88ab-cb81-4ebe-8a79-2cd2fecd9f33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:57.992437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf88ab-cb81-4ebe-8a79-2cd2fecd9f33 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-afcf88ab-cb81-4ebe-8a79-2cd2fecd9f33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:07:58.006728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:07:58.007962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:07:58.008896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:07:58.008896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:07:58.008896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:07:58.008896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:07:58.008896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:07:58.008896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:07:58.008896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:07:58.008896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:07:58.008896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 554/1104] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:07:58.014486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dc1e6-ced5-4449-82cd-9d320330de34 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:58.115821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dc1e6-ced5-4449-82cd-9d320330de34 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:58.145125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:58.151490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae3a02-be4b-4660-a32a-540469822140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:58.152167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae3a02-be4b-4660-a32a-540469822140 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:58.152458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae3a02-be4b-4660-a32a-540469822140 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:58.152734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae3a02-be4b-4660-a32a-540469822140 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:58.153513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae3a02-be4b-4660-a32a-540469822140 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:58.153851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae3a02-be4b-4660-a32a-540469822140 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:58.155280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae3a02-be4b-4660-a32a-540469822140 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:58.168898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 551/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:58.174986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf88ab-cb81-4ebe-8a79-2cd2fecd9f33 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:58.230866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:58.274435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf88ab-cb81-4ebe-8a79-2cd2fecd9f33 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:58.308810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964a287-c344-4f8c-bcd0-f1f981ee76e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:58.309469 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964a287-c344-4f8c-bcd0-f1f981ee76e0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:58.309781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964a287-c344-4f8c-bcd0-f1f981ee76e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:58.310062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964a287-c344-4f8c-bcd0-f1f981ee76e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:58.310895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964a287-c344-4f8c-bcd0-f1f981ee76e0 None admin] RBAC: Adding query filter params (role.id=73f22629793d47f086feb1e90de17023, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=6e6795b5b5cf4499ad494b875756ba71) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:58.311210 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964a287-c344-4f8c-bcd0-f1f981ee76e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:58.312647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964a287-c344-4f8c-bcd0-f1f981ee76e0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:58.318582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 555/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:07:58 2022] GET /identity/v3/role_assignments?user.id=6e6795b5b5cf4499ad494b875756ba71&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=73f22629793d47f086feb1e90de17023 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:59.661620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b79219-36bb-4c24-a1af-823f9247823e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:59.662208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b79219-36bb-4c24-a1af-823f9247823e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:59.662208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b79219-36bb-4c24-a1af-823f9247823e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:59.663571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 552/1107] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:59.669106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ac6ed-a990-4f4e-9d10-0dd5d6c0c77b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:59.669541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ac6ed-a990-4f4e-9d10-0dd5d6c0c77b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:59.669656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ac6ed-a990-4f4e-9d10-0dd5d6c0c77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:59.686059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4c2ac6ed-a990-4f4e-9d10-0dd5d6c0c77b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:59.687169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4c2ac6ed-a990-4f4e-9d10-0dd5d6c0c77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:59.751642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 556/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:07:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:07:59.777273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b79219-36bb-4c24-a1af-823f9247823e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:59.875390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b79219-36bb-4c24-a1af-823f9247823e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:59.909455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0dfb5-1308-424a-9399-c81c7500270d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:59.910264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0dfb5-1308-424a-9399-c81c7500270d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:59.910545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0dfb5-1308-424a-9399-c81c7500270d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:59.910816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0dfb5-1308-424a-9399-c81c7500270d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:59.911975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0dfb5-1308-424a-9399-c81c7500270d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:59.913399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0dfb5-1308-424a-9399-c81c7500270d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:59.915049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 553/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:07:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:59.920899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ac6ed-a990-4f4e-9d10-0dd5d6c0c77b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:59.968932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:00.019680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ac6ed-a990-4f4e-9d10-0dd5d6c0c77b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:00.052156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfc419-f7bc-486f-9e55-eca5f8625bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:00.052824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfc419-f7bc-486f-9e55-eca5f8625bf2 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:00.053106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfc419-f7bc-486f-9e55-eca5f8625bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:00.053420 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfc419-f7bc-486f-9e55-eca5f8625bf2 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:00.054147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfc419-f7bc-486f-9e55-eca5f8625bf2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:00.055593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfc419-f7bc-486f-9e55-eca5f8625bf2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:00.076002 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 557/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:07:59 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:02.339144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e31b21-237d-4977-adf7-c8830397fbc0 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.339792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e31b21-237d-4977-adf7-c8830397fbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.339792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e31b21-237d-4977-adf7-c8830397fbc0 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.341222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 554/1111] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:02.346444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509970b2-2b64-4580-9269-e5ba7201a669 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.346840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509970b2-2b64-4580-9269-e5ba7201a669 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.347053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509970b2-2b64-4580-9269-e5ba7201a669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.364392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-509970b2-2b64-4580-9269-e5ba7201a669 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:02.365582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-509970b2-2b64-4580-9269-e5ba7201a669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:02.432213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 558/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:02.456356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e31b21-237d-4977-adf7-c8830397fbc0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:02.520822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:02.565049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e31b21-237d-4977-adf7-c8830397fbc0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:02.600091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54b92f-7522-46b8-9d7d-5876aa5d899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:02.600866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b92f-7522-46b8-9d7d-5876aa5d899e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.601149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b92f-7522-46b8-9d7d-5876aa5d899e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.601426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b92f-7522-46b8-9d7d-5876aa5d899e None admin] PATH_INFO: `/v3/roles/e7180b44fc2945568dfb83dfaf8b5c79` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.603911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54b92f-7522-46b8-9d7d-5876aa5d899e None admin] RBAC: Authorizing `identity:get_role(role_id=e7180b44fc2945568dfb83dfaf8b5c79)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:02.605308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54b92f-7522-46b8-9d7d-5876aa5d899e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:02.606732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 555/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:02 2022] GET /identity/v3/roles/e7180b44fc2945568dfb83dfaf8b5c79 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:02.612501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509970b2-2b64-4580-9269-e5ba7201a669 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:02.645360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:02.711649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509970b2-2b64-4580-9269-e5ba7201a669 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:02.745523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf9281-2bdc-4934-ac8b-77378a4cd6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:02.746413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf9281-2bdc-4934-ac8b-77378a4cd6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.746697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf9281-2bdc-4934-ac8b-77378a4cd6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.746977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf9281-2bdc-4934-ac8b-77378a4cd6c0 None admin] PATH_INFO: `/v3/users/c4c320de6a43410bba626cd6ef9b6e74` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.759738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf9281-2bdc-4934-ac8b-77378a4cd6c0 None admin] RBAC: Authorizing `identity:get_user(user_id=c4c320de6a43410bba626cd6ef9b6e74)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:02.761196 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf9281-2bdc-4934-ac8b-77378a4cd6c0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:02.762895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 559/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:02 2022] GET /identity/v3/users/c4c320de6a43410bba626cd6ef9b6e74 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:02.768748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54b92f-7522-46b8-9d7d-5876aa5d899e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:02.866997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54b92f-7522-46b8-9d7d-5876aa5d899e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:02.895211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:02.903415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f156a78-da28-467d-9374-c0453c01e868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:02.904068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f156a78-da28-467d-9374-c0453c01e868 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.904354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f156a78-da28-467d-9374-c0453c01e868 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.904633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f156a78-da28-467d-9374-c0453c01e868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.970928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f156a78-da28-467d-9374-c0453c01e868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:02.972380 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f156a78-da28-467d-9374-c0453c01e868 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:03.073911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 556/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:08:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:08:03.079905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf9281-2bdc-4934-ac8b-77378a4cd6c0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:03.169214 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:03.179746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf9281-2bdc-4934-ac8b-77378a4cd6c0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:03.214237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec80bfa-2481-4042-b530-0a50153bcfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:03.215094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec80bfa-2481-4042-b530-0a50153bcfd1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:03.215375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec80bfa-2481-4042-b530-0a50153bcfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:03.215655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec80bfa-2481-4042-b530-0a50153bcfd1 None admin] PATH_INFO: `/v3/projects/7536ba7fce1d4964aadf81b5304f145e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:03.216822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec80bfa-2481-4042-b530-0a50153bcfd1 None admin] RBAC: Authorizing `identity:get_project(project_id=7536ba7fce1d4964aadf81b5304f145e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:03.218280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec80bfa-2481-4042-b530-0a50153bcfd1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:03.219985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 560/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:08:03 2022] GET /identity/v3/projects/7536ba7fce1d4964aadf81b5304f145e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:03.226204 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f156a78-da28-467d-9374-c0453c01e868 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:03.253751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:03.326897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f156a78-da28-467d-9374-c0453c01e868 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:03.360244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69de76-1596-42aa-baa8-d93ce437d688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:03.360905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69de76-1596-42aa-baa8-d93ce437d688 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:03.361183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69de76-1596-42aa-baa8-d93ce437d688 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:03.361457 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69de76-1596-42aa-baa8-d93ce437d688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:03.362302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69de76-1596-42aa-baa8-d93ce437d688 None admin] RBAC: Adding query filter params (role.id=e7180b44fc2945568dfb83dfaf8b5c79, scope.project.id=7536ba7fce1d4964aadf81b5304f145e, user.id=c4c320de6a43410bba626cd6ef9b6e74) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:03.362607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69de76-1596-42aa-baa8-d93ce437d688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:03.364035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69de76-1596-42aa-baa8-d93ce437d688 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:03.370014 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 557/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:08:03 2022] GET /identity/v3/role_assignments?user.id=c4c320de6a43410bba626cd6ef9b6e74&scope.project.id=7536ba7fce1d4964aadf81b5304f145e&role.id=e7180b44fc2945568dfb83dfaf8b5c79 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:05.595272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0943-5647-44c0-a2ce-662f98cc166f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:05.595800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0943-5647-44c0-a2ce-662f98cc166f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:05.595910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0943-5647-44c0-a2ce-662f98cc166f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:05.597213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 561/1118] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:05.602567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e182285-5fb0-4bb5-9fea-59c2a8ed6ad5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:05.603608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e182285-5fb0-4bb5-9fea-59c2a8ed6ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:05.603770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e182285-5fb0-4bb5-9fea-59c2a8ed6ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:05.621476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3e182285-5fb0-4bb5-9fea-59c2a8ed6ad5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:05.622696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3e182285-5fb0-4bb5-9fea-59c2a8ed6ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:05.687316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 558/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:05.710242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c0943-5647-44c0-a2ce-662f98cc166f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:05.814546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c0943-5647-44c0-a2ce-662f98cc166f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:05.847685 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5f7a-bf6c-4a9a-af13-ea1ce98fb6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:05.848546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5f7a-bf6c-4a9a-af13-ea1ce98fb6bf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:05.848833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5f7a-bf6c-4a9a-af13-ea1ce98fb6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:05.849108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5f7a-bf6c-4a9a-af13-ea1ce98fb6bf None admin] PATH_INFO: `/v3/roles/e7180b44fc2945568dfb83dfaf8b5c79` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:05.850605 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd5f7a-bf6c-4a9a-af13-ea1ce98fb6bf None admin] RBAC: Authorizing `identity:get_role(role_id=e7180b44fc2945568dfb83dfaf8b5c79)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:05.851977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd5f7a-bf6c-4a9a-af13-ea1ce98fb6bf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:05.854482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 562/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:05 2022] GET /identity/v3/roles/e7180b44fc2945568dfb83dfaf8b5c79 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:05.860370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e182285-5fb0-4bb5-9fea-59c2a8ed6ad5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:05.884358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:05.959201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e182285-5fb0-4bb5-9fea-59c2a8ed6ad5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:05.992045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f904b-a0bd-45ba-b700-7707b5c101d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:05.992912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f904b-a0bd-45ba-b700-7707b5c101d9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:05.993193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f904b-a0bd-45ba-b700-7707b5c101d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:05.994051 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f904b-a0bd-45ba-b700-7707b5c101d9 None admin] PATH_INFO: `/v3/users/c4c320de6a43410bba626cd6ef9b6e74` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:05.995419 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f904b-a0bd-45ba-b700-7707b5c101d9 None admin] RBAC: Authorizing `identity:get_user(user_id=c4c320de6a43410bba626cd6ef9b6e74)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:05.996886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f904b-a0bd-45ba-b700-7707b5c101d9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:05.998562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 559/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:05 2022] GET /identity/v3/users/c4c320de6a43410bba626cd6ef9b6e74 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:06.004425 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5f7a-bf6c-4a9a-af13-ea1ce98fb6bf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:06.032329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:06.104441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5f7a-bf6c-4a9a-af13-ea1ce98fb6bf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:06.137184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fd44e-96c4-4182-a462-1b3c78e50305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:06.138058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fd44e-96c4-4182-a462-1b3c78e50305 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:06.138343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fd44e-96c4-4182-a462-1b3c78e50305 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:06.138620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fd44e-96c4-4182-a462-1b3c78e50305 None admin] PATH_INFO: `/v3/projects/7536ba7fce1d4964aadf81b5304f145e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:06.139788 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fd44e-96c4-4182-a462-1b3c78e50305 None admin] RBAC: Authorizing `identity:get_project(project_id=7536ba7fce1d4964aadf81b5304f145e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:06.141222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fd44e-96c4-4182-a462-1b3c78e50305 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:06.142947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 563/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:08:06 2022] GET /identity/v3/projects/7536ba7fce1d4964aadf81b5304f145e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:06.149018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f904b-a0bd-45ba-b700-7707b5c101d9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:06.239031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:06.249001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f904b-a0bd-45ba-b700-7707b5c101d9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:06.282304 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae1662-2704-4862-a5e2-6020444ce65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:06.283183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae1662-2704-4862-a5e2-6020444ce65a None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:06.283727 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae1662-2704-4862-a5e2-6020444ce65a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:06.284008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae1662-2704-4862-a5e2-6020444ce65a None admin] PATH_INFO: `/v3/projects/7536ba7fce1d4964aadf81b5304f145e/users/c4c320de6a43410bba626cd6ef9b6e74/roles/e7180b44fc2945568dfb83dfaf8b5c79` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:06.286226 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae1662-2704-4862-a5e2-6020444ce65a None admin] RBAC: Authorizing `identity:create_grant(project_id=7536ba7fce1d4964aadf81b5304f145e, user_id=c4c320de6a43410bba626cd6ef9b6e74, role_id=e7180b44fc2945568dfb83dfaf8b5c79)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:06.287649 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae1662-2704-4862-a5e2-6020444ce65a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:06.300840 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 560/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:08:06 2022] PUT /identity/v3/projects/7536ba7fce1d4964aadf81b5304f145e/users/c4c320de6a43410bba626cd6ef9b6e74/roles/e7180b44fc2945568dfb83dfaf8b5c79 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:08.520994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37a30f-376a-43c9-ad33-0b30c7cc6e29 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:08.521544 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37a30f-376a-43c9-ad33-0b30c7cc6e29 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:08.521544 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37a30f-376a-43c9-ad33-0b30c7cc6e29 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:08.522954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 564/1124] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:08.529015 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42366e1-72a5-4f03-b72c-b0f2f4a7b26e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:08.529474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42366e1-72a5-4f03-b72c-b0f2f4a7b26e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:08.529588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42366e1-72a5-4f03-b72c-b0f2f4a7b26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:08.546883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a42366e1-72a5-4f03-b72c-b0f2f4a7b26e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:08.548072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a42366e1-72a5-4f03-b72c-b0f2f4a7b26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:08.616944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 561/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:08.641793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37a30f-376a-43c9-ad33-0b30c7cc6e29 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:08.735748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:08.742575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37a30f-376a-43c9-ad33-0b30c7cc6e29 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:08.777442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db2779-4a8a-4c81-a6a2-79c7633d621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:08.778318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db2779-4a8a-4c81-a6a2-79c7633d621c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:08.778602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db2779-4a8a-4c81-a6a2-79c7633d621c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:08.778880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db2779-4a8a-4c81-a6a2-79c7633d621c None admin] PATH_INFO: `/v3/roles/e7180b44fc2945568dfb83dfaf8b5c79` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:08.780367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db2779-4a8a-4c81-a6a2-79c7633d621c None admin] RBAC: Authorizing `identity:get_role(role_id=e7180b44fc2945568dfb83dfaf8b5c79)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:08.781760 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db2779-4a8a-4c81-a6a2-79c7633d621c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:08.783148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 565/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:08 2022] GET /identity/v3/roles/e7180b44fc2945568dfb83dfaf8b5c79 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:08.789205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42366e1-72a5-4f03-b72c-b0f2f4a7b26e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:08.868655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:08.889261 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42366e1-72a5-4f03-b72c-b0f2f4a7b26e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:08.923989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cb579-5ab1-4158-bec4-cabe12a67fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:08.924864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cb579-5ab1-4158-bec4-cabe12a67fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:08.925143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cb579-5ab1-4158-bec4-cabe12a67fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:08.925414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cb579-5ab1-4158-bec4-cabe12a67fd4 None admin] PATH_INFO: `/v3/users/c4c320de6a43410bba626cd6ef9b6e74` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:08.926613 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cb579-5ab1-4158-bec4-cabe12a67fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=c4c320de6a43410bba626cd6ef9b6e74)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:08.928031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cb579-5ab1-4158-bec4-cabe12a67fd4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:08.929653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 562/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:08 2022] GET /identity/v3/users/c4c320de6a43410bba626cd6ef9b6e74 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:08.935584 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db2779-4a8a-4c81-a6a2-79c7633d621c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:09.037140 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db2779-4a8a-4c81-a6a2-79c7633d621c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:09.071217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae6f54-028b-4898-a300-f4b45d1a7bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:09.072474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae6f54-028b-4898-a300-f4b45d1a7bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:09.072928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae6f54-028b-4898-a300-f4b45d1a7bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:09.073217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae6f54-028b-4898-a300-f4b45d1a7bb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:09.105595 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:09.143180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae6f54-028b-4898-a300-f4b45d1a7bb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:09.144664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae6f54-028b-4898-a300-f4b45d1a7bb0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:09.245689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 566/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:08:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:08:09.251803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cb579-5ab1-4158-bec4-cabe12a67fd4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:09.367837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cb579-5ab1-4158-bec4-cabe12a67fd4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:09.404326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787268f-eafb-4230-a8fd-ed00b2c29149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:09.405190 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787268f-eafb-4230-a8fd-ed00b2c29149 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:09.405473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787268f-eafb-4230-a8fd-ed00b2c29149 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:09.405779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787268f-eafb-4230-a8fd-ed00b2c29149 None admin] PATH_INFO: `/v3/projects/7536ba7fce1d4964aadf81b5304f145e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:09.406949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787268f-eafb-4230-a8fd-ed00b2c29149 None admin] RBAC: Authorizing `identity:get_project(project_id=7536ba7fce1d4964aadf81b5304f145e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:09.408637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787268f-eafb-4230-a8fd-ed00b2c29149 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:09.410383 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 563/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:08:09 2022] GET /identity/v3/projects/7536ba7fce1d4964aadf81b5304f145e => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:09.417113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae6f54-028b-4898-a300-f4b45d1a7bb0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:09.479984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:09.521315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae6f54-028b-4898-a300-f4b45d1a7bb0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:09.558988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb2be6-ea6b-4714-9955-6ef067f530d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:09.559641 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb2be6-ea6b-4714-9955-6ef067f530d4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:09.559922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb2be6-ea6b-4714-9955-6ef067f530d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:09.560195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb2be6-ea6b-4714-9955-6ef067f530d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:09.561039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb2be6-ea6b-4714-9955-6ef067f530d4 None admin] RBAC: Adding query filter params (role.id=e7180b44fc2945568dfb83dfaf8b5c79, scope.project.id=7536ba7fce1d4964aadf81b5304f145e, user.id=c4c320de6a43410bba626cd6ef9b6e74) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:09.561353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb2be6-ea6b-4714-9955-6ef067f530d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:09.562831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb2be6-ea6b-4714-9955-6ef067f530d4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:09.569261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 567/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:08:09 2022] GET /identity/v3/role_assignments?user.id=c4c320de6a43410bba626cd6ef9b6e74&scope.project.id=7536ba7fce1d4964aadf81b5304f145e&role.id=e7180b44fc2945568dfb83dfaf8b5c79 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:11.810422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1505a1-8701-449d-a8c3-d8f40fe8f527 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:11.810937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1505a1-8701-449d-a8c3-d8f40fe8f527 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:11.811061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1505a1-8701-449d-a8c3-d8f40fe8f527 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:11.812374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 564/1131] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:11.817661 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60237141-eea2-41df-9c76-8c1c68ca8118 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:11.818102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60237141-eea2-41df-9c76-8c1c68ca8118 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:11.818251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60237141-eea2-41df-9c76-8c1c68ca8118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:11.834915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-60237141-eea2-41df-9c76-8c1c68ca8118 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:11.836075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-60237141-eea2-41df-9c76-8c1c68ca8118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:11.902426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 568/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:11.925851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1505a1-8701-449d-a8c3-d8f40fe8f527 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:11.943808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:12.027842 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1505a1-8701-449d-a8c3-d8f40fe8f527 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:12.061275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad1fb0-f115-4439-88df-29ae392aa60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:12.062037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1fb0-f115-4439-88df-29ae392aa60f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:12.062324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1fb0-f115-4439-88df-29ae392aa60f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:12.062601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1fb0-f115-4439-88df-29ae392aa60f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:12.063764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad1fb0-f115-4439-88df-29ae392aa60f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:12.065192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad1fb0-f115-4439-88df-29ae392aa60f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:12.066853 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 565/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:08:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:12.073558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60237141-eea2-41df-9c76-8c1c68ca8118 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:12.129813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:12.171651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60237141-eea2-41df-9c76-8c1c68ca8118 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:12.208974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae90a4-f61d-4cde-9fc7-77df619312f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:12.209635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae90a4-f61d-4cde-9fc7-77df619312f6 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:12.209941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae90a4-f61d-4cde-9fc7-77df619312f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:12.210215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae90a4-f61d-4cde-9fc7-77df619312f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:12.210911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae90a4-f61d-4cde-9fc7-77df619312f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:12.212639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae90a4-f61d-4cde-9fc7-77df619312f6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:12.236807 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 569/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:08:12 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:14.518635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c41d3-d166-4514-9e00-dd969bed905b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:14.519178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c41d3-d166-4514-9e00-dd969bed905b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:14.519178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c41d3-d166-4514-9e00-dd969bed905b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:14.520579 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 566/1135] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:14.526284 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943726ed-3d05-42ce-afd2-78dffc9022c5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:14.526710 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943726ed-3d05-42ce-afd2-78dffc9022c5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:14.526830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943726ed-3d05-42ce-afd2-78dffc9022c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:14.544599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-943726ed-3d05-42ce-afd2-78dffc9022c5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:14.546013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-943726ed-3d05-42ce-afd2-78dffc9022c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:14.618856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 570/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:14.643708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c41d3-d166-4514-9e00-dd969bed905b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:14.728623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:14.745705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c41d3-d166-4514-9e00-dd969bed905b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:14.778332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c891fa-1ea5-4d19-8337-3beaef60fb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:14.779122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c891fa-1ea5-4d19-8337-3beaef60fb2b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:14.779408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c891fa-1ea5-4d19-8337-3beaef60fb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:14.779685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c891fa-1ea5-4d19-8337-3beaef60fb2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:14.780849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c891fa-1ea5-4d19-8337-3beaef60fb2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:14.782309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c891fa-1ea5-4d19-8337-3beaef60fb2b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:14.784184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 567/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:08:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:14.790478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943726ed-3d05-42ce-afd2-78dffc9022c5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:14.831581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:14.889873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943726ed-3d05-42ce-afd2-78dffc9022c5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:14.923885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73b62d-ea3a-49f0-8a04-07f62b334541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:14.924545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73b62d-ea3a-49f0-8a04-07f62b334541 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:14.924824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73b62d-ea3a-49f0-8a04-07f62b334541 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:14.925093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73b62d-ea3a-49f0-8a04-07f62b334541 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:14.925817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73b62d-ea3a-49f0-8a04-07f62b334541 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:14.927251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73b62d-ea3a-49f0-8a04-07f62b334541 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:14.947709 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 571/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:08:14 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:17.201483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2be02-5003-45af-aab2-18e38dfc8870 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.202037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2be02-5003-45af-aab2-18e38dfc8870 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.202123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2be02-5003-45af-aab2-18e38dfc8870 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.203471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 568/1139] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:17.209108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aa6d0-5936-4594-8101-cb0306d98f8c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.209574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aa6d0-5936-4594-8101-cb0306d98f8c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.209691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aa6d0-5936-4594-8101-cb0306d98f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.226977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c11aa6d0-5936-4594-8101-cb0306d98f8c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:17.228453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c11aa6d0-5936-4594-8101-cb0306d98f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:17.293442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 572/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:17.317325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2be02-5003-45af-aab2-18e38dfc8870 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.417587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2be02-5003-45af-aab2-18e38dfc8870 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.453995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c806e9-b3af-47c2-a63a-cce280ea6cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.454718 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c806e9-b3af-47c2-a63a-cce280ea6cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.455002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c806e9-b3af-47c2-a63a-cce280ea6cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.455277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c806e9-b3af-47c2-a63a-cce280ea6cf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.465506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c806e9-b3af-47c2-a63a-cce280ea6cf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-78c806e9-b3af-47c2-a63a-cce280ea6cf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:08:17.473136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:08:17.474258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:08:17.475281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 569/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:08:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:08:17.480691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11aa6d0-5936-4594-8101-cb0306d98f8c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.507400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:17.580494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11aa6d0-5936-4594-8101-cb0306d98f8c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.613882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4ed20-a3fb-4dd1-a3ea-fd5d83e0c87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.614549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4ed20-a3fb-4dd1-a3ea-fd5d83e0c87b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.614834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4ed20-a3fb-4dd1-a3ea-fd5d83e0c87b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.615110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4ed20-a3fb-4dd1-a3ea-fd5d83e0c87b None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.615857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4ed20-a3fb-4dd1-a3ea-fd5d83e0c87b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:17.616166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4ed20-a3fb-4dd1-a3ea-fd5d83e0c87b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.617534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4ed20-a3fb-4dd1-a3ea-fd5d83e0c87b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:17.626044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 573/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:08:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:17.631947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c806e9-b3af-47c2-a63a-cce280ea6cf9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.647015 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:17.730075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c806e9-b3af-47c2-a63a-cce280ea6cf9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.762698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bafe2e-5e61-48d6-ba71-c96dfb4df512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.763559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bafe2e-5e61-48d6-ba71-c96dfb4df512 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.763842 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bafe2e-5e61-48d6-ba71-c96dfb4df512 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.764119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bafe2e-5e61-48d6-ba71-c96dfb4df512 None admin] PATH_INFO: `/v3/users/069a66e940ad4c939eb9e86461d6beb2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.777023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bafe2e-5e61-48d6-ba71-c96dfb4df512 None admin] RBAC: Authorizing `identity:get_user(user_id=069a66e940ad4c939eb9e86461d6beb2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.778476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bafe2e-5e61-48d6-ba71-c96dfb4df512 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:17.780122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 570/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:17 2022] GET /identity/v3/users/069a66e940ad4c939eb9e86461d6beb2 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:17.786493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4ed20-a3fb-4dd1-a3ea-fd5d83e0c87b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.873302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:17.885528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4ed20-a3fb-4dd1-a3ea-fd5d83e0c87b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.919228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc13f0a-117b-4aea-b5f1-fe5bb93cd6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.919873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc13f0a-117b-4aea-b5f1-fe5bb93cd6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.920155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc13f0a-117b-4aea-b5f1-fe5bb93cd6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.920429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc13f0a-117b-4aea-b5f1-fe5bb93cd6a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.988732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc13f0a-117b-4aea-b5f1-fe5bb93cd6a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.990137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc13f0a-117b-4aea-b5f1-fe5bb93cd6a3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:18.075831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:18.090844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 574/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:08:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:08:18.096746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bafe2e-5e61-48d6-ba71-c96dfb4df512 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:18.164416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:18.195153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bafe2e-5e61-48d6-ba71-c96dfb4df512 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:18.228915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12113bdd-9bdf-4434-8fd6-767828303ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:18.229788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12113bdd-9bdf-4434-8fd6-767828303ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:18.230069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12113bdd-9bdf-4434-8fd6-767828303ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:18.230336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12113bdd-9bdf-4434-8fd6-767828303ba3 None admin] PATH_INFO: `/v3/projects/aab55a3ac2184bc1a1d0572eeb4b8ba0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:18.231489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12113bdd-9bdf-4434-8fd6-767828303ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=aab55a3ac2184bc1a1d0572eeb4b8ba0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:18.232906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12113bdd-9bdf-4434-8fd6-767828303ba3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:18.234895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 571/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:08:18 2022] GET /identity/v3/projects/aab55a3ac2184bc1a1d0572eeb4b8ba0 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:18.241049 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc13f0a-117b-4aea-b5f1-fe5bb93cd6a3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:18.343214 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc13f0a-117b-4aea-b5f1-fe5bb93cd6a3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:18.377153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041231c8-9f36-4c6a-84e3-a190a8c19e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:18.377822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041231c8-9f36-4c6a-84e3-a190a8c19e02 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:18.378102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041231c8-9f36-4c6a-84e3-a190a8c19e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:18.378401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041231c8-9f36-4c6a-84e3-a190a8c19e02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:18.379218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041231c8-9f36-4c6a-84e3-a190a8c19e02 None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=aab55a3ac2184bc1a1d0572eeb4b8ba0, user.id=069a66e940ad4c939eb9e86461d6beb2) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:18.379526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041231c8-9f36-4c6a-84e3-a190a8c19e02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:18.380946 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041231c8-9f36-4c6a-84e3-a190a8c19e02 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:18.387094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 575/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:08:18 2022] GET /identity/v3/role_assignments?user.id=069a66e940ad4c939eb9e86461d6beb2&scope.project.id=aab55a3ac2184bc1a1d0572eeb4b8ba0&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:20.600681 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903618d-0618-49ae-a566-249dbad6fc63 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.601305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903618d-0618-49ae-a566-249dbad6fc63 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.601305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903618d-0618-49ae-a566-249dbad6fc63 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.603313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 572/1147] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:20.608566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a625-6ce9-4ba1-9bbf-fe3efd334f93 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.609014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a625-6ce9-4ba1-9bbf-fe3efd334f93 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.609116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a625-6ce9-4ba1-9bbf-fe3efd334f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.629602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5741a625-6ce9-4ba1-9bbf-fe3efd334f93 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:20.630832 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5741a625-6ce9-4ba1-9bbf-fe3efd334f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:20.676938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:20.696990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 576/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:20.719718 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903618d-0618-49ae-a566-249dbad6fc63 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:20.822444 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903618d-0618-49ae-a566-249dbad6fc63 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:20.856837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f225e-6508-4d0b-a005-04cd5f410f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:20.857615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f225e-6508-4d0b-a005-04cd5f410f39 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.857922 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f225e-6508-4d0b-a005-04cd5f410f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.858195 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f225e-6508-4d0b-a005-04cd5f410f39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.865062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:20.869421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f225e-6508-4d0b-a005-04cd5f410f39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-330f225e-6508-4d0b-a005-04cd5f410f39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:08:20.877568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:08:20.878771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:08:20.879888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 573/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:08:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:08:20.885399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741a625-6ce9-4ba1-9bbf-fe3efd334f93 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:20.981484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741a625-6ce9-4ba1-9bbf-fe3efd334f93 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:21.015452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15659f-c478-4608-9441-e45cdcefe60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:21.016179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15659f-c478-4608-9441-e45cdcefe60c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:21.016453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15659f-c478-4608-9441-e45cdcefe60c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:21.016726 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15659f-c478-4608-9441-e45cdcefe60c None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:21.017470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15659f-c478-4608-9441-e45cdcefe60c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:21.017804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15659f-c478-4608-9441-e45cdcefe60c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:21.019168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15659f-c478-4608-9441-e45cdcefe60c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:21.027681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 577/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:08:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:21.033879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f225e-6508-4d0b-a005-04cd5f410f39 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:21.132406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f225e-6508-4d0b-a005-04cd5f410f39 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:21.166079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0f62e-a293-4b1d-ba64-c739fa09b7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:21.166941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0f62e-a293-4b1d-ba64-c739fa09b7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:21.167222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0f62e-a293-4b1d-ba64-c739fa09b7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:21.167495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0f62e-a293-4b1d-ba64-c739fa09b7a4 None admin] PATH_INFO: `/v3/users/069a66e940ad4c939eb9e86461d6beb2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:21.168698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0f62e-a293-4b1d-ba64-c739fa09b7a4 None admin] RBAC: Authorizing `identity:get_user(user_id=069a66e940ad4c939eb9e86461d6beb2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:21.170175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0f62e-a293-4b1d-ba64-c739fa09b7a4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:21.171812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 574/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:21 2022] GET /identity/v3/users/069a66e940ad4c939eb9e86461d6beb2 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:21.177613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15659f-c478-4608-9441-e45cdcefe60c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:21.222892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:21.277370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15659f-c478-4608-9441-e45cdcefe60c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:21.310241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31abc65-b473-4c38-a4db-21dfd772e65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:21.311093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31abc65-b473-4c38-a4db-21dfd772e65e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:21.311378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31abc65-b473-4c38-a4db-21dfd772e65e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:21.311655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31abc65-b473-4c38-a4db-21dfd772e65e None admin] PATH_INFO: `/v3/projects/aab55a3ac2184bc1a1d0572eeb4b8ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:21.312826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31abc65-b473-4c38-a4db-21dfd772e65e None admin] RBAC: Authorizing `identity:get_project(project_id=aab55a3ac2184bc1a1d0572eeb4b8ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:21.314297 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31abc65-b473-4c38-a4db-21dfd772e65e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:21.316003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 578/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:08:21 2022] GET /identity/v3/projects/aab55a3ac2184bc1a1d0572eeb4b8ba0 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:21.322720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0f62e-a293-4b1d-ba64-c739fa09b7a4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:21.387568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:21.422989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0f62e-a293-4b1d-ba64-c739fa09b7a4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:21.456901 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d15799-5c1d-4e43-93f2-0145e1e1384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:21.457832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d15799-5c1d-4e43-93f2-0145e1e1384c None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:21.458119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d15799-5c1d-4e43-93f2-0145e1e1384c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:21.458398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d15799-5c1d-4e43-93f2-0145e1e1384c None admin] PATH_INFO: `/v3/projects/aab55a3ac2184bc1a1d0572eeb4b8ba0/users/069a66e940ad4c939eb9e86461d6beb2/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:21.460472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d15799-5c1d-4e43-93f2-0145e1e1384c None admin] RBAC: Authorizing `identity:create_grant(project_id=aab55a3ac2184bc1a1d0572eeb4b8ba0, user_id=069a66e940ad4c939eb9e86461d6beb2, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:21.462781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d15799-5c1d-4e43-93f2-0145e1e1384c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:21.476572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 575/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:08:21 2022] PUT /identity/v3/projects/aab55a3ac2184bc1a1d0572eeb4b8ba0/users/069a66e940ad4c939eb9e86461d6beb2/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:23.688290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc8317-1fc5-4ce0-a148-78bc49979157 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:23.688849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc8317-1fc5-4ce0-a148-78bc49979157 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:23.688849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc8317-1fc5-4ce0-a148-78bc49979157 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:23.690207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 579/1154] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:23.695539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64694f0c-9e15-470c-8bd2-7e453ab7d33f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:23.696726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64694f0c-9e15-470c-8bd2-7e453ab7d33f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:23.696726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64694f0c-9e15-470c-8bd2-7e453ab7d33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:23.713745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-64694f0c-9e15-470c-8bd2-7e453ab7d33f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:23.714916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-64694f0c-9e15-470c-8bd2-7e453ab7d33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:23.778745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 576/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:23.801894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc8317-1fc5-4ce0-a148-78bc49979157 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:23.900365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc8317-1fc5-4ce0-a148-78bc49979157 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:23.913309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:23.934227 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae538d-d5ff-4d73-a859-ebc12551dd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:23.935003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae538d-d5ff-4d73-a859-ebc12551dd88 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:23.935281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae538d-d5ff-4d73-a859-ebc12551dd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:23.935552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae538d-d5ff-4d73-a859-ebc12551dd88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:23.945345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae538d-d5ff-4d73-a859-ebc12551dd88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f9ae538d-d5ff-4d73-a859-ebc12551dd88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:08:23.954043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:08:23.955166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:08:23.956371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 580/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:08:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:08:23.961280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64694f0c-9e15-470c-8bd2-7e453ab7d33f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:24.017040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:24.062339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64694f0c-9e15-470c-8bd2-7e453ab7d33f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:24.095976 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d51ea4-a515-46d1-9ac4-862d43c83158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:24.096634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51ea4-a515-46d1-9ac4-862d43c83158 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:24.096916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51ea4-a515-46d1-9ac4-862d43c83158 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:24.097190 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51ea4-a515-46d1-9ac4-862d43c83158 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:24.098108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d51ea4-a515-46d1-9ac4-862d43c83158 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:24.098417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d51ea4-a515-46d1-9ac4-862d43c83158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:24.099766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d51ea4-a515-46d1-9ac4-862d43c83158 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:24.108003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 577/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:24.113855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae538d-d5ff-4d73-a859-ebc12551dd88 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:24.215060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae538d-d5ff-4d73-a859-ebc12551dd88 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:24.248133 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a934ffd-0993-4a04-aec8-2b9ff3dd44ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:24.249005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a934ffd-0993-4a04-aec8-2b9ff3dd44ce None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:24.249287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a934ffd-0993-4a04-aec8-2b9ff3dd44ce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:24.249567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a934ffd-0993-4a04-aec8-2b9ff3dd44ce None admin] PATH_INFO: `/v3/users/069a66e940ad4c939eb9e86461d6beb2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:24.250778 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a934ffd-0993-4a04-aec8-2b9ff3dd44ce None admin] RBAC: Authorizing `identity:get_user(user_id=069a66e940ad4c939eb9e86461d6beb2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:24.252230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a934ffd-0993-4a04-aec8-2b9ff3dd44ce None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:24.253894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 581/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:24 2022] GET /identity/v3/users/069a66e940ad4c939eb9e86461d6beb2 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:24.260557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d51ea4-a515-46d1-9ac4-862d43c83158 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:24.359098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d51ea4-a515-46d1-9ac4-862d43c83158 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:24.375927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:24.392474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5ce21-4de6-445a-bce7-083f9a216e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:24.393121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5ce21-4de6-445a-bce7-083f9a216e9a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:24.393408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5ce21-4de6-445a-bce7-083f9a216e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:24.393685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5ce21-4de6-445a-bce7-083f9a216e9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:24.460109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5ce21-4de6-445a-bce7-083f9a216e9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:24.461569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5ce21-4de6-445a-bce7-083f9a216e9a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:24.560110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 578/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:08:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:08:24.566119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a934ffd-0993-4a04-aec8-2b9ff3dd44ce None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:24.608778 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:24.665784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a934ffd-0993-4a04-aec8-2b9ff3dd44ce None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:24.702938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d871ad-9242-4968-a3e6-2601cfd02964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:24.703804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d871ad-9242-4968-a3e6-2601cfd02964 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:24.704087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d871ad-9242-4968-a3e6-2601cfd02964 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:24.704364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d871ad-9242-4968-a3e6-2601cfd02964 None admin] PATH_INFO: `/v3/projects/aab55a3ac2184bc1a1d0572eeb4b8ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:24.705545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d871ad-9242-4968-a3e6-2601cfd02964 None admin] RBAC: Authorizing `identity:get_project(project_id=aab55a3ac2184bc1a1d0572eeb4b8ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:24.707005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d871ad-9242-4968-a3e6-2601cfd02964 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:24.708724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 582/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:08:24 2022] GET /identity/v3/projects/aab55a3ac2184bc1a1d0572eeb4b8ba0 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:24.715054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5ce21-4de6-445a-bce7-083f9a216e9a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:24.733735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:24.813781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5ce21-4de6-445a-bce7-083f9a216e9a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:24.846862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9091d-8fbd-4ee0-bf0b-c2437e89b723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:24.847509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9091d-8fbd-4ee0-bf0b-c2437e89b723 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:24.847792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9091d-8fbd-4ee0-bf0b-c2437e89b723 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:24.848067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9091d-8fbd-4ee0-bf0b-c2437e89b723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:24.848880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9091d-8fbd-4ee0-bf0b-c2437e89b723 None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=aab55a3ac2184bc1a1d0572eeb4b8ba0, user.id=069a66e940ad4c939eb9e86461d6beb2) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:24.849192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9091d-8fbd-4ee0-bf0b-c2437e89b723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:24.850639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9091d-8fbd-4ee0-bf0b-c2437e89b723 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:24.857001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 579/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:08:24 2022] GET /identity/v3/role_assignments?user.id=069a66e940ad4c939eb9e86461d6beb2&scope.project.id=aab55a3ac2184bc1a1d0572eeb4b8ba0&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:27.087034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88d7ac-f0da-4c6b-886a-43e41398abe1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.087851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88d7ac-f0da-4c6b-886a-43e41398abe1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.087851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88d7ac-f0da-4c6b-886a-43e41398abe1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.088972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 583/1162] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:27.094398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f89aa6-9fb5-4fd2-aebd-3e2aac2700ca None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.094801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f89aa6-9fb5-4fd2-aebd-3e2aac2700ca None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.094964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f89aa6-9fb5-4fd2-aebd-3e2aac2700ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.112251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-41f89aa6-9fb5-4fd2-aebd-3e2aac2700ca None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:27.113429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-41f89aa6-9fb5-4fd2-aebd-3e2aac2700ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:27.180476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 580/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:27.203085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88d7ac-f0da-4c6b-886a-43e41398abe1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:27.303976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88d7ac-f0da-4c6b-886a-43e41398abe1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:27.313124 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:27.338130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e19ba-ccbe-4201-82d0-ab78d1241307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:27.338708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e19ba-ccbe-4201-82d0-ab78d1241307 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.338993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e19ba-ccbe-4201-82d0-ab78d1241307 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.339272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e19ba-ccbe-4201-82d0-ab78d1241307 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.405179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e19ba-ccbe-4201-82d0-ab78d1241307 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:27.406665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e19ba-ccbe-4201-82d0-ab78d1241307 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:27.504684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 584/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:08:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:08:27.511365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f89aa6-9fb5-4fd2-aebd-3e2aac2700ca None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:27.525655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:27.613041 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f89aa6-9fb5-4fd2-aebd-3e2aac2700ca None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:27.646251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8241c-8615-4403-a88e-aa67a4e868b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:27.647054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8241c-8615-4403-a88e-aa67a4e868b7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.647336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8241c-8615-4403-a88e-aa67a4e868b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.647607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8241c-8615-4403-a88e-aa67a4e868b7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.655089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8241c-8615-4403-a88e-aa67a4e868b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:27.656521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8241c-8615-4403-a88e-aa67a4e868b7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-cec8241c-8615-4403-a88e-aa67a4e868b7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:08:27.670773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:08:27.672020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:08:27.672919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 581/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Wed May 25 16:08:27 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:08:27.678115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e19ba-ccbe-4201-82d0-ab78d1241307 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:27.688204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:27.776191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e19ba-ccbe-4201-82d0-ab78d1241307 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:27.809814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9ea63-dd18-4b11-b841-13e117a2c622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:27.810475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9ea63-dd18-4b11-b841-13e117a2c622 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.810757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9ea63-dd18-4b11-b841-13e117a2c622 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.811035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9ea63-dd18-4b11-b841-13e117a2c622 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.811751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9ea63-dd18-4b11-b841-13e117a2c622 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:27.812059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9ea63-dd18-4b11-b841-13e117a2c622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:27.813429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9ea63-dd18-4b11-b841-13e117a2c622 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:27.820583 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 585/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Wed May 25 16:08:27 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:30.051785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0c2b-7d0d-4df5-9848-b27f0ff17f87 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.052350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0c2b-7d0d-4df5-9848-b27f0ff17f87 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.052350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0c2b-7d0d-4df5-9848-b27f0ff17f87 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.053710 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 582/1167] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:30.059833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32d8d0-c002-4917-9b98-f078e3fa7cd9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.060253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32d8d0-c002-4917-9b98-f078e3fa7cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.060398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32d8d0-c002-4917-9b98-f078e3fa7cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.077356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3e32d8d0-c002-4917-9b98-f078e3fa7cd9 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:30.078572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3e32d8d0-c002-4917-9b98-f078e3fa7cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:30.144041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 586/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:30.171866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b0c2b-7d0d-4df5-9848-b27f0ff17f87 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.243275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:30.276556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b0c2b-7d0d-4df5-9848-b27f0ff17f87 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:30.313490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916745a-b18e-4b6d-a806-74c895e90cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:30.314247 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916745a-b18e-4b6d-a806-74c895e90cff None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.314534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916745a-b18e-4b6d-a806-74c895e90cff None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.314813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916745a-b18e-4b6d-a806-74c895e90cff None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.315548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916745a-b18e-4b6d-a806-74c895e90cff None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:30.316903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916745a-b18e-4b6d-a806-74c895e90cff None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.331878 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 583/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Wed May 25 16:08:30 2022] POST /identity/v3/domains => generated 267 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:32.568652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da593c3-e62e-47f1-890c-99dd6d3dbcc3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.569203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da593c3-e62e-47f1-890c-99dd6d3dbcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.569203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da593c3-e62e-47f1-890c-99dd6d3dbcc3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.570894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 587/1170] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:32.576232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6b8fc-0459-4ef9-a51f-8401d262f78f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.576693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6b8fc-0459-4ef9-a51f-8401d262f78f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.576786 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6b8fc-0459-4ef9-a51f-8401d262f78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.594119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e5b6b8fc-0459-4ef9-a51f-8401d262f78f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:32.595286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e5b6b8fc-0459-4ef9-a51f-8401d262f78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:32.663370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 584/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:32.686877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da593c3-e62e-47f1-890c-99dd6d3dbcc3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:32.689738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:32.790604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da593c3-e62e-47f1-890c-99dd6d3dbcc3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:32.824200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc4e27-cc05-4f8b-a6bf-4077d6235861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:32.825053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc4e27-cc05-4f8b-a6bf-4077d6235861 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.825335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc4e27-cc05-4f8b-a6bf-4077d6235861 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.825609 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc4e27-cc05-4f8b-a6bf-4077d6235861 None admin] PATH_INFO: `/v3/domains/69b3c38e686c44ca9bd1b4a5928be8e1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.835728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc4e27-cc05-4f8b-a6bf-4077d6235861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=69b3c38e686c44ca9bd1b4a5928be8e1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:32.837869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc4e27-cc05-4f8b-a6bf-4077d6235861 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:32.839503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 588/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Wed May 25 16:08:32 2022] GET /identity/v3/domains/69b3c38e686c44ca9bd1b4a5928be8e1 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:32.845352 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6b8fc-0459-4ef9-a51f-8401d262f78f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:32.905256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:32.943870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6b8fc-0459-4ef9-a51f-8401d262f78f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:32.977471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac3975-1560-484a-95a0-4dba92025c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:32.978165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3975-1560-484a-95a0-4dba92025c19 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.978454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3975-1560-484a-95a0-4dba92025c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.978737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3975-1560-484a-95a0-4dba92025c19 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.979438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac3975-1560-484a-95a0-4dba92025c19 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:32.980860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac3975-1560-484a-95a0-4dba92025c19 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:33.005944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 585/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:08:32 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:35.251959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2c1ed-32d4-4cce-9f5f-44347b00492e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.252511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2c1ed-32d4-4cce-9f5f-44347b00492e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.252511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2c1ed-32d4-4cce-9f5f-44347b00492e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.253946 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 589/1174] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:35.259191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446df3-7d36-448f-a82d-f1115de405aa None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.259586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446df3-7d36-448f-a82d-f1115de405aa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.259745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446df3-7d36-448f-a82d-f1115de405aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.276798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-21446df3-7d36-448f-a82d-f1115de405aa None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:35.278023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-21446df3-7d36-448f-a82d-f1115de405aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:35.344477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 586/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:35.373589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2c1ed-32d4-4cce-9f5f-44347b00492e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.438740 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:35.478360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2c1ed-32d4-4cce-9f5f-44347b00492e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.510995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e28931-010c-48b7-87ab-baba8fefa228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.511774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e28931-010c-48b7-87ab-baba8fefa228 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.512054 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e28931-010c-48b7-87ab-baba8fefa228 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.512329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e28931-010c-48b7-87ab-baba8fefa228 None admin] PATH_INFO: `/v3/domains/69b3c38e686c44ca9bd1b4a5928be8e1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.513498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e28931-010c-48b7-87ab-baba8fefa228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=69b3c38e686c44ca9bd1b4a5928be8e1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.514961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e28931-010c-48b7-87ab-baba8fefa228 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.516578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 590/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Wed May 25 16:08:35 2022] GET /identity/v3/domains/69b3c38e686c44ca9bd1b4a5928be8e1 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:35.522569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21446df3-7d36-448f-a82d-f1115de405aa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.568708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:35.624280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21446df3-7d36-448f-a82d-f1115de405aa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.659092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c9898-a380-4b89-8d26-22b034c69480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.659754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9898-a380-4b89-8d26-22b034c69480 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.660036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9898-a380-4b89-8d26-22b034c69480 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.660314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9898-a380-4b89-8d26-22b034c69480 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.661023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c9898-a380-4b89-8d26-22b034c69480 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.662474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c9898-a380-4b89-8d26-22b034c69480 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.681726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 587/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:38.163152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae06ec9-015f-40bc-800c-84b21286b388 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:38.163687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae06ec9-015f-40bc-800c-84b21286b388 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:38.163687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae06ec9-015f-40bc-800c-84b21286b388 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:38.165807 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 591/1178] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:38.171200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5773e5a-fe72-4ede-b601-039e7ff537ae None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:38.171627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5773e5a-fe72-4ede-b601-039e7ff537ae None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:38.171753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5773e5a-fe72-4ede-b601-039e7ff537ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:38.189064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a5773e5a-fe72-4ede-b601-039e7ff537ae None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:38.190278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a5773e5a-fe72-4ede-b601-039e7ff537ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:38.255067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 588/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:38.278804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae06ec9-015f-40bc-800c-84b21286b388 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:38.378886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae06ec9-015f-40bc-800c-84b21286b388 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:38.406934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:38.412861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba73ed-f6d7-4f27-be95-8d71dc4a2691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:38.413737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba73ed-f6d7-4f27-be95-8d71dc4a2691 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:38.414617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba73ed-f6d7-4f27-be95-8d71dc4a2691 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:38.415035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba73ed-f6d7-4f27-be95-8d71dc4a2691 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:38.424890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba73ed-f6d7-4f27-be95-8d71dc4a2691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0cba73ed-f6d7-4f27-be95-8d71dc4a2691 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:08:38.433282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:08:38.434426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:08:38.435477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 592/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:08:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:08:38.442526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5773e5a-fe72-4ede-b601-039e7ff537ae None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:38.471850 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:38.544457 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5773e5a-fe72-4ede-b601-039e7ff537ae None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:38.577857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6baf7-177a-4fe2-9e57-51032f05cab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:38.578525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6baf7-177a-4fe2-9e57-51032f05cab8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:38.578809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6baf7-177a-4fe2-9e57-51032f05cab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:38.579082 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6baf7-177a-4fe2-9e57-51032f05cab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:38.579823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6baf7-177a-4fe2-9e57-51032f05cab8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:38.580127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6baf7-177a-4fe2-9e57-51032f05cab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:38.581484 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6baf7-177a-4fe2-9e57-51032f05cab8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:38.590345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 589/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:08:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:38.596167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba73ed-f6d7-4f27-be95-8d71dc4a2691 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:38.695786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba73ed-f6d7-4f27-be95-8d71dc4a2691 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:38.728915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8eba82-d7b5-45b7-9562-ed546bff6adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:38.729813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8eba82-d7b5-45b7-9562-ed546bff6adc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:38.730099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8eba82-d7b5-45b7-9562-ed546bff6adc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:38.730370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8eba82-d7b5-45b7-9562-ed546bff6adc None admin] PATH_INFO: `/v3/users/d98655d702714ab08b01b10e707cb48a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:38.744046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8eba82-d7b5-45b7-9562-ed546bff6adc None admin] RBAC: Authorizing `identity:get_user(user_id=d98655d702714ab08b01b10e707cb48a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:38.745487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8eba82-d7b5-45b7-9562-ed546bff6adc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:38.747162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 593/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:38 2022] GET /identity/v3/users/d98655d702714ab08b01b10e707cb48a => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:38.753009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6baf7-177a-4fe2-9e57-51032f05cab8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:38.843948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:38.853725 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6baf7-177a-4fe2-9e57-51032f05cab8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:38.886398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11b56f-0699-49e7-a4a3-2d7d1f6fd3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:38.887039 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11b56f-0699-49e7-a4a3-2d7d1f6fd3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:38.887324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11b56f-0699-49e7-a4a3-2d7d1f6fd3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:38.887601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11b56f-0699-49e7-a4a3-2d7d1f6fd3a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:38.954247 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11b56f-0699-49e7-a4a3-2d7d1f6fd3a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:38.955665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11b56f-0699-49e7-a4a3-2d7d1f6fd3a7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:39.044264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:39.054184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 590/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:08:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:08:39.060583 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8eba82-d7b5-45b7-9562-ed546bff6adc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:39.109465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:39.160148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8eba82-d7b5-45b7-9562-ed546bff6adc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:39.193450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ae63a-6cf1-4cd6-bc25-e841966b2dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:39.194337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ae63a-6cf1-4cd6-bc25-e841966b2dae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:39.194619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ae63a-6cf1-4cd6-bc25-e841966b2dae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:39.194897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ae63a-6cf1-4cd6-bc25-e841966b2dae None admin] PATH_INFO: `/v3/projects/648d8869d8f946dc8cae3fe45a6fb6c6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:39.196728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ae63a-6cf1-4cd6-bc25-e841966b2dae None admin] RBAC: Authorizing `identity:get_project(project_id=648d8869d8f946dc8cae3fe45a6fb6c6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:39.198208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ae63a-6cf1-4cd6-bc25-e841966b2dae None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:39.199936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 594/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:08:39 2022] GET /identity/v3/projects/648d8869d8f946dc8cae3fe45a6fb6c6 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:39.206559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11b56f-0699-49e7-a4a3-2d7d1f6fd3a7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:39.303386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11b56f-0699-49e7-a4a3-2d7d1f6fd3a7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:39.337768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c91677-0ea5-4beb-b10d-bcdf1ec18d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:39.338421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c91677-0ea5-4beb-b10d-bcdf1ec18d68 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:39.338709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c91677-0ea5-4beb-b10d-bcdf1ec18d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:39.338988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c91677-0ea5-4beb-b10d-bcdf1ec18d68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:39.339807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c91677-0ea5-4beb-b10d-bcdf1ec18d68 None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=648d8869d8f946dc8cae3fe45a6fb6c6, user.id=d98655d702714ab08b01b10e707cb48a) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:39.340120 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c91677-0ea5-4beb-b10d-bcdf1ec18d68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:39.341533 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c91677-0ea5-4beb-b10d-bcdf1ec18d68 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:39.346970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 591/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:08:39 2022] GET /identity/v3/role_assignments?user.id=d98655d702714ab08b01b10e707cb48a&scope.project.id=648d8869d8f946dc8cae3fe45a6fb6c6&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:41.548860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48de6c-6e45-49db-950e-eda2c7ef05b6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.549429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48de6c-6e45-49db-950e-eda2c7ef05b6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.549429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48de6c-6e45-49db-950e-eda2c7ef05b6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.550844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 595/1186] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:41.556461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55c9e0-9f16-4ce9-807b-d53e5056bdf3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.556917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55c9e0-9f16-4ce9-807b-d53e5056bdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.557019 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55c9e0-9f16-4ce9-807b-d53e5056bdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.574144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9e55c9e0-9f16-4ce9-807b-d53e5056bdf3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:41.575348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9e55c9e0-9f16-4ce9-807b-d53e5056bdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:41.626288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:41.642838 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 592/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 16:08:41.665874 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48de6c-6e45-49db-950e-eda2c7ef05b6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:41.770983 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48de6c-6e45-49db-950e-eda2c7ef05b6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:41.786976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:41.805493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577dc167-c37c-452d-bd18-f1954cf26a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:41.806327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dc167-c37c-452d-bd18-f1954cf26a4f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.806611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dc167-c37c-452d-bd18-f1954cf26a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.806882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dc167-c37c-452d-bd18-f1954cf26a4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.816739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577dc167-c37c-452d-bd18-f1954cf26a4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-577dc167-c37c-452d-bd18-f1954cf26a4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:08:41.824327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:08:41.825451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:08:41.826670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 596/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:08:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:08:41.831665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55c9e0-9f16-4ce9-807b-d53e5056bdf3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:41.930465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55c9e0-9f16-4ce9-807b-d53e5056bdf3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:41.963171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f86567-67b7-4dcd-930a-5c91d92c94a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:41.963834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f86567-67b7-4dcd-930a-5c91d92c94a6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.964117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f86567-67b7-4dcd-930a-5c91d92c94a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.964391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f86567-67b7-4dcd-930a-5c91d92c94a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.965127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f86567-67b7-4dcd-930a-5c91d92c94a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:41.965433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f86567-67b7-4dcd-930a-5c91d92c94a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:41.966826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f86567-67b7-4dcd-930a-5c91d92c94a6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:41.974743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 593/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:08:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:41.981528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577dc167-c37c-452d-bd18-f1954cf26a4f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:42.086056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577dc167-c37c-452d-bd18-f1954cf26a4f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:42.118794 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1d685-834c-4ebb-a7b8-a418bd2c16f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:42.119663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1d685-834c-4ebb-a7b8-a418bd2c16f4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:42.119942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1d685-834c-4ebb-a7b8-a418bd2c16f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:42.120215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1d685-834c-4ebb-a7b8-a418bd2c16f4 None admin] PATH_INFO: `/v3/users/d98655d702714ab08b01b10e707cb48a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:42.121382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1d685-834c-4ebb-a7b8-a418bd2c16f4 None admin] RBAC: Authorizing `identity:get_user(user_id=d98655d702714ab08b01b10e707cb48a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:42.122851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1d685-834c-4ebb-a7b8-a418bd2c16f4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:42.124457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 597/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:41 2022] GET /identity/v3/users/d98655d702714ab08b01b10e707cb48a => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:42.130280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f86567-67b7-4dcd-930a-5c91d92c94a6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:42.179373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:42.233822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f86567-67b7-4dcd-930a-5c91d92c94a6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:42.266761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0963bc-a432-4a08-bd10-e6d547905ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:42.267625 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0963bc-a432-4a08-bd10-e6d547905ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:42.267910 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0963bc-a432-4a08-bd10-e6d547905ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:42.268191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0963bc-a432-4a08-bd10-e6d547905ad7 None admin] PATH_INFO: `/v3/projects/648d8869d8f946dc8cae3fe45a6fb6c6` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:42.269361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0963bc-a432-4a08-bd10-e6d547905ad7 None admin] RBAC: Authorizing `identity:get_project(project_id=648d8869d8f946dc8cae3fe45a6fb6c6)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:42.270822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0963bc-a432-4a08-bd10-e6d547905ad7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:42.272517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 594/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:08:42 2022] GET /identity/v3/projects/648d8869d8f946dc8cae3fe45a6fb6c6 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:42.279357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1d685-834c-4ebb-a7b8-a418bd2c16f4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:42.326865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:42.380991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1d685-834c-4ebb-a7b8-a418bd2c16f4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:42.415029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f931fda-2139-4c3e-8ef1-21184016942d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:42.415911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f931fda-2139-4c3e-8ef1-21184016942d None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:42.416197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f931fda-2139-4c3e-8ef1-21184016942d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:42.416477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f931fda-2139-4c3e-8ef1-21184016942d None admin] PATH_INFO: `/v3/projects/648d8869d8f946dc8cae3fe45a6fb6c6/users/d98655d702714ab08b01b10e707cb48a/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:42.418548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f931fda-2139-4c3e-8ef1-21184016942d None admin] RBAC: Authorizing `identity:create_grant(project_id=648d8869d8f946dc8cae3fe45a6fb6c6, user_id=d98655d702714ab08b01b10e707cb48a, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:42.419986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f931fda-2139-4c3e-8ef1-21184016942d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:42.433457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 598/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:08:42 2022] PUT /identity/v3/projects/648d8869d8f946dc8cae3fe45a6fb6c6/users/d98655d702714ab08b01b10e707cb48a/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:44.643422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74144f79-987a-4f86-bf6a-86dc726415dc None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.643944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74144f79-987a-4f86-bf6a-86dc726415dc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.644056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74144f79-987a-4f86-bf6a-86dc726415dc None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.645362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 595/1193] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:44.650882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd214c-6faa-4df2-8156-5477fa2828be None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.651300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd214c-6faa-4df2-8156-5477fa2828be None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.651440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd214c-6faa-4df2-8156-5477fa2828be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.672183 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8bcd214c-6faa-4df2-8156-5477fa2828be None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:44.673383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8bcd214c-6faa-4df2-8156-5477fa2828be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:44.743396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 599/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:08:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:08:44.766886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74144f79-987a-4f86-bf6a-86dc726415dc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.871529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74144f79-987a-4f86-bf6a-86dc726415dc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:44.886998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:44.905175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7e9e4-e11a-43f1-abf5-9b5d82c75ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:44.905998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7e9e4-e11a-43f1-abf5-9b5d82c75ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.906287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7e9e4-e11a-43f1-abf5-9b5d82c75ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.906563 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7e9e4-e11a-43f1-abf5-9b5d82c75ed3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.917889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7e9e4-e11a-43f1-abf5-9b5d82c75ed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1cd7e9e4-e11a-43f1-abf5-9b5d82c75ed3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:08:44.925426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:08:44.926599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:08:44.927496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 596/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:08:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:08:44.933413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd214c-6faa-4df2-8156-5477fa2828be None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.966358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:45.041824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd214c-6faa-4df2-8156-5477fa2828be None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.075224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae226253-e668-4794-b620-46adec786d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.075887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae226253-e668-4794-b620-46adec786d53 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.076168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae226253-e668-4794-b620-46adec786d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.076446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae226253-e668-4794-b620-46adec786d53 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.077186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae226253-e668-4794-b620-46adec786d53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:45.077495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae226253-e668-4794-b620-46adec786d53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.078893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae226253-e668-4794-b620-46adec786d53 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.087395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 600/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:08:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:45.093245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7e9e4-e11a-43f1-abf5-9b5d82c75ed3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.191376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7e9e4-e11a-43f1-abf5-9b5d82c75ed3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.224584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a04904-64bb-4329-939e-b4e634d2db35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.225451 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a04904-64bb-4329-939e-b4e634d2db35 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.225758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a04904-64bb-4329-939e-b4e634d2db35 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.226613 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a04904-64bb-4329-939e-b4e634d2db35 None admin] PATH_INFO: `/v3/users/d98655d702714ab08b01b10e707cb48a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.227802 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a04904-64bb-4329-939e-b4e634d2db35 None admin] RBAC: Authorizing `identity:get_user(user_id=d98655d702714ab08b01b10e707cb48a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.229227 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a04904-64bb-4329-939e-b4e634d2db35 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.230919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 597/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Wed May 25 16:08:45 2022] GET /identity/v3/users/d98655d702714ab08b01b10e707cb48a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:45.236686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae226253-e668-4794-b620-46adec786d53 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.329802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:45.337036 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae226253-e668-4794-b620-46adec786d53 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.372631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea0c27-5570-400b-bee4-66471ae0fbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.373273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea0c27-5570-400b-bee4-66471ae0fbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.373558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea0c27-5570-400b-bee4-66471ae0fbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.373863 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea0c27-5570-400b-bee4-66471ae0fbd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.441706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea0c27-5570-400b-bee4-66471ae0fbd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.443201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea0c27-5570-400b-bee4-66471ae0fbd8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.534057 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:45.541053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 601/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:08:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:08:45.546852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a04904-64bb-4329-939e-b4e634d2db35 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.595618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:45.651128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a04904-64bb-4329-939e-b4e634d2db35 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.685727 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12419b5-74dc-468b-8cc3-141a18752080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.686608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12419b5-74dc-468b-8cc3-141a18752080 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.686893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12419b5-74dc-468b-8cc3-141a18752080 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.687170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12419b5-74dc-468b-8cc3-141a18752080 None admin] PATH_INFO: `/v3/projects/648d8869d8f946dc8cae3fe45a6fb6c6` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.688339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12419b5-74dc-468b-8cc3-141a18752080 None admin] RBAC: Authorizing `identity:get_project(project_id=648d8869d8f946dc8cae3fe45a6fb6c6)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.689834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12419b5-74dc-468b-8cc3-141a18752080 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.691562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 598/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Wed May 25 16:08:45 2022] GET /identity/v3/projects/648d8869d8f946dc8cae3fe45a6fb6c6 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:45.698374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea0c27-5570-400b-bee4-66471ae0fbd8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.795960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea0c27-5570-400b-bee4-66471ae0fbd8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.829991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c601d1a-002d-46cd-98fb-d23d54ee554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.830636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c601d1a-002d-46cd-98fb-d23d54ee554f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.830915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c601d1a-002d-46cd-98fb-d23d54ee554f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.831184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c601d1a-002d-46cd-98fb-d23d54ee554f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.832006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c601d1a-002d-46cd-98fb-d23d54ee554f None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=648d8869d8f946dc8cae3fe45a6fb6c6, user.id=d98655d702714ab08b01b10e707cb48a) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:45.832313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c601d1a-002d-46cd-98fb-d23d54ee554f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.833750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c601d1a-002d-46cd-98fb-d23d54ee554f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.839778 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 602/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:08:45 2022] GET /identity/v3/role_assignments?user.id=d98655d702714ab08b01b10e707cb48a&scope.project.id=648d8869d8f946dc8cae3fe45a6fb6c6&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:55.681511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=44 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:55.837234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=70 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:47.013767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b25845-796a-4885-b6e9-99d6e0e71b47 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:47.013767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b25845-796a-4885-b6e9-99d6e0e71b47 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:47.013767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b25845-796a-4885-b6e9-99d6e0e71b47 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:47.014900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 599/1201] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:09:47.020395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c825c22-1aab-4d60-8ed5-0d31661dd413 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:47.020961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c825c22-1aab-4d60-8ed5-0d31661dd413 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:47.020961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c825c22-1aab-4d60-8ed5-0d31661dd413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:47.041017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2c825c22-1aab-4d60-8ed5-0d31661dd413 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:47.042795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2c825c22-1aab-4d60-8ed5-0d31661dd413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:47.147677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 603/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:09:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:09:47.173408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b25845-796a-4885-b6e9-99d6e0e71b47 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:47.322064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b25845-796a-4885-b6e9-99d6e0e71b47 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:47.373009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171def5-6804-464d-bfbc-105bdd886d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:47.373915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171def5-6804-464d-bfbc-105bdd886d26 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:47.374204 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171def5-6804-464d-bfbc-105bdd886d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:47.374497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171def5-6804-464d-bfbc-105bdd886d26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:47.397824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171def5-6804-464d-bfbc-105bdd886d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:47.399681 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171def5-6804-464d-bfbc-105bdd886d26 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4171def5-6804-464d-bfbc-105bdd886d26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:09:47.423364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:09:47.424468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:09:47.425458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:09:47.425458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:09:47.425458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:09:47.425458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:09:47.425458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 600/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:09:47.434897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c825c22-1aab-4d60-8ed5-0d31661dd413 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:47.573731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c825c22-1aab-4d60-8ed5-0d31661dd413 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:47.609945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:47.618477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69207b9-c5cd-4226-837b-2468926a3335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:47.619279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69207b9-c5cd-4226-837b-2468926a3335 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:47.619578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69207b9-c5cd-4226-837b-2468926a3335 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:47.619890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69207b9-c5cd-4226-837b-2468926a3335 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:47.620883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69207b9-c5cd-4226-837b-2468926a3335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:47.621020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69207b9-c5cd-4226-837b-2468926a3335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:47.622372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69207b9-c5cd-4226-837b-2468926a3335 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:47.636170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 604/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:47.642268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171def5-6804-464d-bfbc-105bdd886d26 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:47.717863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:47.791071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171def5-6804-464d-bfbc-105bdd886d26 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:47.830807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0816b7-29dc-401d-9554-34f6acf648f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:47.831741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0816b7-29dc-401d-9554-34f6acf648f6 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:47.832178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0816b7-29dc-401d-9554-34f6acf648f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:47.832744 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0816b7-29dc-401d-9554-34f6acf648f6 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:47.833865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0816b7-29dc-401d-9554-34f6acf648f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:47.835401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0816b7-29dc-401d-9554-34f6acf648f6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:47.862571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 601/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Wed May 25 16:09:47 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:09:50.186469 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6fef-708b-47a9-9a55-a94400dc4039 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:50.187114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6fef-708b-47a9-9a55-a94400dc4039 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:50.187114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6fef-708b-47a9-9a55-a94400dc4039 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:50.188640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 605/1206] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:09:50.196450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801549b-ff9b-4bf8-a2db-8cd53504c693 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:50.196450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801549b-ff9b-4bf8-a2db-8cd53504c693 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:50.196450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801549b-ff9b-4bf8-a2db-8cd53504c693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:50.213539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a801549b-ff9b-4bf8-a2db-8cd53504c693 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:50.214746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a801549b-ff9b-4bf8-a2db-8cd53504c693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:50.298879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 602/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:09:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:09:50.326929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f6fef-708b-47a9-9a55-a94400dc4039 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:50.466305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f6fef-708b-47a9-9a55-a94400dc4039 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:50.507214 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bde336-30dd-4b49-b008-3d037bcd457e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:50.508086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bde336-30dd-4b49-b008-3d037bcd457e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:50.508435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bde336-30dd-4b49-b008-3d037bcd457e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:50.508705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bde336-30dd-4b49-b008-3d037bcd457e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:50.520349 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bde336-30dd-4b49-b008-3d037bcd457e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-05bde336-30dd-4b49-b008-3d037bcd457e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:09:50.528852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:09:50.530237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:09:50.531119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:09:50.535119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 606/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:09:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:09:50.539088 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801549b-ff9b-4bf8-a2db-8cd53504c693 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:50.586672 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:50.658772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801549b-ff9b-4bf8-a2db-8cd53504c693 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:50.702504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447315b-7867-4e73-9ef0-474c6538ac8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:50.703245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447315b-7867-4e73-9ef0-474c6538ac8c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:50.703535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447315b-7867-4e73-9ef0-474c6538ac8c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:50.703756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447315b-7867-4e73-9ef0-474c6538ac8c None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:50.704551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447315b-7867-4e73-9ef0-474c6538ac8c None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:50.705025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447315b-7867-4e73-9ef0-474c6538ac8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:50.706455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447315b-7867-4e73-9ef0-474c6538ac8c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:50.715862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 603/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:09:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:50.721762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bde336-30dd-4b49-b008-3d037bcd457e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:50.787282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:50.855654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bde336-30dd-4b49-b008-3d037bcd457e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:50.901853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f43823-3ff5-4f68-a758-e65cc737c8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:50.902721 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f43823-3ff5-4f68-a758-e65cc737c8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:50.903009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f43823-3ff5-4f68-a758-e65cc737c8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:50.903326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f43823-3ff5-4f68-a758-e65cc737c8a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:50.915884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f43823-3ff5-4f68-a758-e65cc737c8a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:50.917491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f43823-3ff5-4f68-a758-e65cc737c8a5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-67f43823-3ff5-4f68-a758-e65cc737c8a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:09:50.937706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:09:50.939099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:09:50.940270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:09:50.940270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:09:50.940270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:09:50.940270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:09:50.944895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 607/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:09:50.946296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447315b-7867-4e73-9ef0-474c6538ac8c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:51.070981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:51.075058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447315b-7867-4e73-9ef0-474c6538ac8c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:51.119551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b39d8f-1c0b-4f71-a95f-737e4c914eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:51.120242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b39d8f-1c0b-4f71-a95f-737e4c914eac None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:51.120528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b39d8f-1c0b-4f71-a95f-737e4c914eac None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:51.120814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b39d8f-1c0b-4f71-a95f-737e4c914eac None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:51.121531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b39d8f-1c0b-4f71-a95f-737e4c914eac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:51.121868 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b39d8f-1c0b-4f71-a95f-737e4c914eac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:51.123387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b39d8f-1c0b-4f71-a95f-737e4c914eac None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:51.146390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 604/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:51.148563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f43823-3ff5-4f68-a758-e65cc737c8a5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:51.272860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f43823-3ff5-4f68-a758-e65cc737c8a5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:51.274858 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:51.313359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f22ec8-ab6b-4bab-8992-5de7af0540f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:51.314336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f22ec8-ab6b-4bab-8992-5de7af0540f3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:51.314989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f22ec8-ab6b-4bab-8992-5de7af0540f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:51.315385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f22ec8-ab6b-4bab-8992-5de7af0540f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:51.324213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f22ec8-ab6b-4bab-8992-5de7af0540f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:51.326167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f22ec8-ab6b-4bab-8992-5de7af0540f3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-15f22ec8-ab6b-4bab-8992-5de7af0540f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:09:51.343601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:09:51.344662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:09:51.345651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:09:51.345651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:09:51.345651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:09:51.345651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:09:51.345651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:09:51.345651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:09:51.345651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:09:51.345651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:09:51.345651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 16:09:51.345651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:09:51.351706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 608/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:09:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:09:51.351884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b39d8f-1c0b-4f71-a95f-737e4c914eac None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:51.479522 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b39d8f-1c0b-4f71-a95f-737e4c914eac None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:51.523242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c132427-c769-4e97-bf55-993db78a13ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:51.524087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c132427-c769-4e97-bf55-993db78a13ca None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:51.524373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c132427-c769-4e97-bf55-993db78a13ca None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:51.524783 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c132427-c769-4e97-bf55-993db78a13ca None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:51.525816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c132427-c769-4e97-bf55-993db78a13ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:51.526131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c132427-c769-4e97-bf55-993db78a13ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:51.527804 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c132427-c769-4e97-bf55-993db78a13ca None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:51.549747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 605/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Wed May 25 16:09:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:51.556197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f22ec8-ab6b-4bab-8992-5de7af0540f3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:51.701550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f22ec8-ab6b-4bab-8992-5de7af0540f3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:51.760496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4220bfa-801a-49de-878f-236ee1f6c75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:51.761501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4220bfa-801a-49de-878f-236ee1f6c75d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:51.761962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4220bfa-801a-49de-878f-236ee1f6c75d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:51.762378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4220bfa-801a-49de-878f-236ee1f6c75d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:51.811031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:51.866395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4220bfa-801a-49de-878f-236ee1f6c75d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:51.867859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4220bfa-801a-49de-878f-236ee1f6c75d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:52.004110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 609/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:09:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:09:52.010433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c132427-c769-4e97-bf55-993db78a13ca None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:52.034800 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:52.136610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c132427-c769-4e97-bf55-993db78a13ca None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:52.185610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f3002-8c03-4cc9-8971-93d417f99e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:52.186470 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f3002-8c03-4cc9-8971-93d417f99e27 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:52.186751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f3002-8c03-4cc9-8971-93d417f99e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:52.187025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f3002-8c03-4cc9-8971-93d417f99e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:52.201453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f3002-8c03-4cc9-8971-93d417f99e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:52.203020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f3002-8c03-4cc9-8971-93d417f99e27 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6f7f3002-8c03-4cc9-8971-93d417f99e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:09:52.228027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:09:52.229000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:09:52.229915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:09:52.229915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:09:52.229915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:09:52.229915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:09:52.230911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 606/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:09:52.237382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4220bfa-801a-49de-878f-236ee1f6c75d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:52.305602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:52.355980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4220bfa-801a-49de-878f-236ee1f6c75d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:52.398881 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4301c-9e64-4dc2-a5fa-ef7b900f23e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:52.399555 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4301c-9e64-4dc2-a5fa-ef7b900f23e6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:52.399840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4301c-9e64-4dc2-a5fa-ef7b900f23e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:52.400107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4301c-9e64-4dc2-a5fa-ef7b900f23e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:52.400832 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4301c-9e64-4dc2-a5fa-ef7b900f23e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:52.401152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4301c-9e64-4dc2-a5fa-ef7b900f23e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:52.402563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4301c-9e64-4dc2-a5fa-ef7b900f23e6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:52.420185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 610/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:52.421558 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f3002-8c03-4cc9-8971-93d417f99e27 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:52.498734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:52.543140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f3002-8c03-4cc9-8971-93d417f99e27 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:52.584058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de88d80-a79b-4f97-a605-d7ee78843762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:52.585141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88d80-a79b-4f97-a605-d7ee78843762 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:52.585484 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88d80-a79b-4f97-a605-d7ee78843762 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:52.585791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88d80-a79b-4f97-a605-d7ee78843762 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:52.593816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de88d80-a79b-4f97-a605-d7ee78843762 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:52.595473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de88d80-a79b-4f97-a605-d7ee78843762 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8de88d80-a79b-4f97-a605-d7ee78843762 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:09:52.613089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:09:52.615124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:09:52.620913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:09:52.620913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:09:52.620913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:09:52.620913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:09:52.620913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:09:52.620913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:09:52.620913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:09:52.620913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:09:52.620913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 607/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:09:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:09:52.621440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4301c-9e64-4dc2-a5fa-ef7b900f23e6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:52.757450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4301c-9e64-4dc2-a5fa-ef7b900f23e6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:52.784605 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:52.798624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4906bc-8717-4ca5-9920-e3a896b8e9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:52.799400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4906bc-8717-4ca5-9920-e3a896b8e9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:52.799663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4906bc-8717-4ca5-9920-e3a896b8e9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:52.799977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4906bc-8717-4ca5-9920-e3a896b8e9a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:52.800744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4906bc-8717-4ca5-9920-e3a896b8e9a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:52.801652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4906bc-8717-4ca5-9920-e3a896b8e9a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:52.803186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4906bc-8717-4ca5-9920-e3a896b8e9a8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:52.818833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 611/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:09:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:52.825704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de88d80-a79b-4f97-a605-d7ee78843762 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:52.942904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de88d80-a79b-4f97-a605-d7ee78843762 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:52.978268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:52.986688 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a936d4-8888-4c98-a2c9-6f0563deaf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:52.987639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a936d4-8888-4c98-a2c9-6f0563deaf9f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:52.988102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a936d4-8888-4c98-a2c9-6f0563deaf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:52.989063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a936d4-8888-4c98-a2c9-6f0563deaf9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:52.990096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a936d4-8888-4c98-a2c9-6f0563deaf9f None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=6719421144734ef49b3809df30a45961) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:52.990543 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a936d4-8888-4c98-a2c9-6f0563deaf9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:52.992080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a936d4-8888-4c98-a2c9-6f0563deaf9f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:52.998531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 608/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:09:52 2022] GET /identity/v3/role_assignments?user.id=6719421144734ef49b3809df30a45961&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:55.271282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9892003-2892-4ff2-99e2-ba45a9928fca None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:55.271861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9892003-2892-4ff2-99e2-ba45a9928fca None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:55.271861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9892003-2892-4ff2-99e2-ba45a9928fca None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:55.273984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 612/1220] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:09:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:09:55.281508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718cf90-ed06-4106-8ce7-ae0a83741e55 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:55.281508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718cf90-ed06-4106-8ce7-ae0a83741e55 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:55.281508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718cf90-ed06-4106-8ce7-ae0a83741e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:55.300236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5718cf90-ed06-4106-8ce7-ae0a83741e55 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:55.302018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5718cf90-ed06-4106-8ce7-ae0a83741e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:55.382752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 609/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:09:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:09:55.407730 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9892003-2892-4ff2-99e2-ba45a9928fca None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:55.537706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9892003-2892-4ff2-99e2-ba45a9928fca None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:55.589266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67564d7-754a-4817-ab3d-d2212259624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:55.590193 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67564d7-754a-4817-ab3d-d2212259624b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:55.590471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67564d7-754a-4817-ab3d-d2212259624b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:55.590742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67564d7-754a-4817-ab3d-d2212259624b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:55.602754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67564d7-754a-4817-ab3d-d2212259624b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c67564d7-754a-4817-ab3d-d2212259624b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:09:55.611080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:09:55.612318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:09:55.613289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 613/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:09:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:09:55.618898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718cf90-ed06-4106-8ce7-ae0a83741e55 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:55.733196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718cf90-ed06-4106-8ce7-ae0a83741e55 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:55.760338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:55.772342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df898651-40a0-4810-a2de-3d7f2c2b747a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:55.773032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898651-40a0-4810-a2de-3d7f2c2b747a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:55.773316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898651-40a0-4810-a2de-3d7f2c2b747a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:55.773597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898651-40a0-4810-a2de-3d7f2c2b747a None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:55.774414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df898651-40a0-4810-a2de-3d7f2c2b747a None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:55.774716 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df898651-40a0-4810-a2de-3d7f2c2b747a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:55.776336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df898651-40a0-4810-a2de-3d7f2c2b747a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:55.786692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 610/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:09:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:55.793829 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67564d7-754a-4817-ab3d-d2212259624b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:55.849427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:55.914424 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67564d7-754a-4817-ab3d-d2212259624b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:55.953617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347c0fe-931d-4262-9de7-e58a4d875707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:55.954464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347c0fe-931d-4262-9de7-e58a4d875707 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:55.954749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347c0fe-931d-4262-9de7-e58a4d875707 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:55.955026 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347c0fe-931d-4262-9de7-e58a4d875707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:55.965908 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347c0fe-931d-4262-9de7-e58a4d875707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:55.967365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347c0fe-931d-4262-9de7-e58a4d875707 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8347c0fe-931d-4262-9de7-e58a4d875707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:09:55.985793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:09:55.986789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:09:55.994879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:09:55.994879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:09:55.994879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:09:55.994879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:09:55.994879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 614/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:09:55.995248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df898651-40a0-4810-a2de-3d7f2c2b747a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:56.119637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df898651-40a0-4810-a2de-3d7f2c2b747a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:56.163594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92950c-9968-425e-9625-86907bff0188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:56.164663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92950c-9968-425e-9625-86907bff0188 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:56.164951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92950c-9968-425e-9625-86907bff0188 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:56.165745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92950c-9968-425e-9625-86907bff0188 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:56.166051 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92950c-9968-425e-9625-86907bff0188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:56.166361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92950c-9968-425e-9625-86907bff0188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:56.167740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92950c-9968-425e-9625-86907bff0188 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:56.185356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 611/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:56.192959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347c0fe-931d-4262-9de7-e58a4d875707 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:56.312994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:56.320950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347c0fe-931d-4262-9de7-e58a4d875707 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:56.356437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c57bfd-a6b9-4029-8a4d-c1eff3319de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:56.357245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c57bfd-a6b9-4029-8a4d-c1eff3319de2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:56.357530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c57bfd-a6b9-4029-8a4d-c1eff3319de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:56.357839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c57bfd-a6b9-4029-8a4d-c1eff3319de2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:56.365690 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c57bfd-a6b9-4029-8a4d-c1eff3319de2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:56.367163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c57bfd-a6b9-4029-8a4d-c1eff3319de2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-79c57bfd-a6b9-4029-8a4d-c1eff3319de2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:09:56.381860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:09:56.383028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:09:56.388708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:09:56.388708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:09:56.388708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:09:56.388708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:09:56.388708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:09:56.388708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:09:56.388708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:09:56.388708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:09:56.388708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 16:09:56.388708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:09:56.388708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 615/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:09:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:09:56.389340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92950c-9968-425e-9625-86907bff0188 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:56.445812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:56.502522 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92950c-9968-425e-9625-86907bff0188 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:56.541634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826c51f-7f17-4c5a-99f8-25327c378178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:56.542357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826c51f-7f17-4c5a-99f8-25327c378178 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:56.542651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826c51f-7f17-4c5a-99f8-25327c378178 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:56.542906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826c51f-7f17-4c5a-99f8-25327c378178 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:56.543781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826c51f-7f17-4c5a-99f8-25327c378178 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:56.544090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826c51f-7f17-4c5a-99f8-25327c378178 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:56.545551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826c51f-7f17-4c5a-99f8-25327c378178 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:56.568355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 612/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Wed May 25 16:09:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:56.576182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c57bfd-a6b9-4029-8a4d-c1eff3319de2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:56.692656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c57bfd-a6b9-4029-8a4d-c1eff3319de2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:56.731318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88f541-5137-45e2-b395-0040a2f84463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:56.732125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88f541-5137-45e2-b395-0040a2f84463 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:56.732396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88f541-5137-45e2-b395-0040a2f84463 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:56.732666 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88f541-5137-45e2-b395-0040a2f84463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:56.744676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88f541-5137-45e2-b395-0040a2f84463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:56.746361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88f541-5137-45e2-b395-0040a2f84463 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4c88f541-5137-45e2-b395-0040a2f84463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:09:56.765005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:09:56.766253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:09:56.767370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:09:56.767370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:09:56.767370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:09:56.767370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:09:56.767370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 616/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:09:56.773157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826c51f-7f17-4c5a-99f8-25327c378178 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:56.888849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826c51f-7f17-4c5a-99f8-25327c378178 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:56.891322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:56.931777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788e407-2d40-48f7-99bc-dda138886707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:56.932531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788e407-2d40-48f7-99bc-dda138886707 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:56.932855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788e407-2d40-48f7-99bc-dda138886707 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:56.933162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788e407-2d40-48f7-99bc-dda138886707 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:56.933980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788e407-2d40-48f7-99bc-dda138886707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:56.934328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788e407-2d40-48f7-99bc-dda138886707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:56.935786 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788e407-2d40-48f7-99bc-dda138886707 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:56.949514 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 613/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:56.956298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88f541-5137-45e2-b395-0040a2f84463 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:56.966363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:57.069845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88f541-5137-45e2-b395-0040a2f84463 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:57.107871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38347fc3-3f6a-418b-a77a-ca637bde700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:57.108721 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38347fc3-3f6a-418b-a77a-ca637bde700a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:57.109036 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38347fc3-3f6a-418b-a77a-ca637bde700a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:57.109314 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38347fc3-3f6a-418b-a77a-ca637bde700a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:57.116803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38347fc3-3f6a-418b-a77a-ca637bde700a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:57.118389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38347fc3-3f6a-418b-a77a-ca637bde700a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-38347fc3-3f6a-418b-a77a-ca637bde700a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:09:57.134522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:09:57.135738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:09:57.139009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:09:57.139009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:09:57.139009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:09:57.139009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:09:57.139009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:09:57.139009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:09:57.139009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:09:57.139009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:09:57.139009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 617/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:09:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:09:57.141873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788e407-2d40-48f7-99bc-dda138886707 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:57.257327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788e407-2d40-48f7-99bc-dda138886707 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:57.306308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33077c3a-d52f-4485-813d-62f2f54fd5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:57.307662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33077c3a-d52f-4485-813d-62f2f54fd5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:57.308166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33077c3a-d52f-4485-813d-62f2f54fd5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:57.308569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33077c3a-d52f-4485-813d-62f2f54fd5f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:57.309581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33077c3a-d52f-4485-813d-62f2f54fd5f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:57.310099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33077c3a-d52f-4485-813d-62f2f54fd5f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:57.312241 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33077c3a-d52f-4485-813d-62f2f54fd5f7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:57.329201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 614/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:09:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:09:57.336362 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38347fc3-3f6a-418b-a77a-ca637bde700a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:57.414738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:57.451963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38347fc3-3f6a-418b-a77a-ca637bde700a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:57.489133 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fed10-ea7e-4ea6-b125-e44616cf2dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:57.490120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fed10-ea7e-4ea6-b125-e44616cf2dca None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:57.490403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fed10-ea7e-4ea6-b125-e44616cf2dca None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:57.490681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fed10-ea7e-4ea6-b125-e44616cf2dca None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/6719421144734ef49b3809df30a45961/roles/6bc01dbf83c844fe802466d3c4ada6af` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:57.505683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fed10-ea7e-4ea6-b125-e44616cf2dca None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=6719421144734ef49b3809df30a45961, role_id=6bc01dbf83c844fe802466d3c4ada6af)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:57.507175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fed10-ea7e-4ea6-b125-e44616cf2dca None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:57.521259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 618/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:09:57 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/6719421144734ef49b3809df30a45961/roles/6bc01dbf83c844fe802466d3c4ada6af => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:59.996920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4006e16-366f-499c-a69d-e99aaf5addb4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.997471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4006e16-366f-499c-a69d-e99aaf5addb4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.997471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4006e16-366f-499c-a69d-e99aaf5addb4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.998857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 615/1233] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:10:00.004890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661386-79e3-4890-91bb-c0aebbbb73b0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:00.005464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661386-79e3-4890-91bb-c0aebbbb73b0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:00.005464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661386-79e3-4890-91bb-c0aebbbb73b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:00.023949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-44661386-79e3-4890-91bb-c0aebbbb73b0 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:00.025252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-44661386-79e3-4890-91bb-c0aebbbb73b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:00.100345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 619/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:10:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:10:00.126338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4006e16-366f-499c-a69d-e99aaf5addb4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:00.155623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:00.253450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4006e16-366f-499c-a69d-e99aaf5addb4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:00.292628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5cc767-6db0-4972-bdb2-47a9e5954487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:00.293485 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5cc767-6db0-4972-bdb2-47a9e5954487 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:00.293875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5cc767-6db0-4972-bdb2-47a9e5954487 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:00.294162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5cc767-6db0-4972-bdb2-47a9e5954487 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:00.308048 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5cc767-6db0-4972-bdb2-47a9e5954487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3c5cc767-6db0-4972-bdb2-47a9e5954487 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:10:00.316565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:00.317903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:00.319244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 616/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Wed May 25 16:10:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 16:10:00.324884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44661386-79e3-4890-91bb-c0aebbbb73b0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:00.381013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:00.441825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44661386-79e3-4890-91bb-c0aebbbb73b0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:00.478026 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe8ebc-21c1-4194-ad94-14010d6c8026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:00.478831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe8ebc-21c1-4194-ad94-14010d6c8026 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:00.479161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe8ebc-21c1-4194-ad94-14010d6c8026 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:00.479437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe8ebc-21c1-4194-ad94-14010d6c8026 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:00.480192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe8ebc-21c1-4194-ad94-14010d6c8026 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:00.480535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe8ebc-21c1-4194-ad94-14010d6c8026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:00.482190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe8ebc-21c1-4194-ad94-14010d6c8026 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:00.491685 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 620/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Wed May 25 16:10:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:00.497520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5cc767-6db0-4972-bdb2-47a9e5954487 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:00.595821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:00.613994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5cc767-6db0-4972-bdb2-47a9e5954487 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:00.655101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f375832-dcc4-4972-9e10-690961e449f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:00.656088 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f375832-dcc4-4972-9e10-690961e449f6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:00.656229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f375832-dcc4-4972-9e10-690961e449f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:00.656490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f375832-dcc4-4972-9e10-690961e449f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:00.667944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f375832-dcc4-4972-9e10-690961e449f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:00.669469 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f375832-dcc4-4972-9e10-690961e449f6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9f375832-dcc4-4972-9e10-690961e449f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:10:00.687621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:00.688823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:00.689799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:10:00.689799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:10:00.689799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:00.689799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:00.689799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 617/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:00.695183 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe8ebc-21c1-4194-ad94-14010d6c8026 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:00.811458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe8ebc-21c1-4194-ad94-14010d6c8026 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:00.823944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:00.855755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973523f-f787-4fd8-b60e-4ea7a8a3cc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:00.856350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973523f-f787-4fd8-b60e-4ea7a8a3cc07 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:00.856636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973523f-f787-4fd8-b60e-4ea7a8a3cc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:00.856913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973523f-f787-4fd8-b60e-4ea7a8a3cc07 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:00.857642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973523f-f787-4fd8-b60e-4ea7a8a3cc07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:00.857979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973523f-f787-4fd8-b60e-4ea7a8a3cc07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:00.859651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973523f-f787-4fd8-b60e-4ea7a8a3cc07 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:00.875751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 621/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:00.882371 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f375832-dcc4-4972-9e10-690961e449f6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:01.001297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f375832-dcc4-4972-9e10-690961e449f6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:01.040152 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20514367-05f0-42a9-ba72-1c8cad64f373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:01.040955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20514367-05f0-42a9-ba72-1c8cad64f373 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:01.041232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20514367-05f0-42a9-ba72-1c8cad64f373 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:01.041499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20514367-05f0-42a9-ba72-1c8cad64f373 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:01.044324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:01.050080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20514367-05f0-42a9-ba72-1c8cad64f373 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:01.052391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20514367-05f0-42a9-ba72-1c8cad64f373 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-20514367-05f0-42a9-ba72-1c8cad64f373 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:10:01.069495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:01.070645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:01.071627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:01.071627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:01.071627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:10:01.071627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:10:01.071627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:10:01.071627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:10:01.071627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:10:01.071627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:10:01.071627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 16:10:01.071627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:01.071627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 618/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:01.076572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973523f-f787-4fd8-b60e-4ea7a8a3cc07 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:01.211072 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973523f-f787-4fd8-b60e-4ea7a8a3cc07 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:01.263248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bfa40-b97e-4f24-8e6a-9554d39ab99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:01.264179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bfa40-b97e-4f24-8e6a-9554d39ab99b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:01.264618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bfa40-b97e-4f24-8e6a-9554d39ab99b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:01.264782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bfa40-b97e-4f24-8e6a-9554d39ab99b None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:01.265978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bfa40-b97e-4f24-8e6a-9554d39ab99b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:01.266360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bfa40-b97e-4f24-8e6a-9554d39ab99b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:01.268899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bfa40-b97e-4f24-8e6a-9554d39ab99b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:01.292530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 622/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Wed May 25 16:10:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:01.298718 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20514367-05f0-42a9-ba72-1c8cad64f373 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:01.410400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20514367-05f0-42a9-ba72-1c8cad64f373 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:01.448804 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe62c9-97de-4b46-9cf5-f597fef728d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:01.449457 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe62c9-97de-4b46-9cf5-f597fef728d4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:01.449765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe62c9-97de-4b46-9cf5-f597fef728d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:01.450044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe62c9-97de-4b46-9cf5-f597fef728d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:01.531762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:01.542867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe62c9-97de-4b46-9cf5-f597fef728d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:01.544333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe62c9-97de-4b46-9cf5-f597fef728d4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:01.664988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 619/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:10:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:10:01.670681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bfa40-b97e-4f24-8e6a-9554d39ab99b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:01.702094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:01.791316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bfa40-b97e-4f24-8e6a-9554d39ab99b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:01.829063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c151c2-c0e4-43fa-af6e-a2be79f2d7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:01.830141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c151c2-c0e4-43fa-af6e-a2be79f2d7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:01.830435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c151c2-c0e4-43fa-af6e-a2be79f2d7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:01.830773 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c151c2-c0e4-43fa-af6e-a2be79f2d7b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:01.842912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c151c2-c0e4-43fa-af6e-a2be79f2d7b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:01.844392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c151c2-c0e4-43fa-af6e-a2be79f2d7b4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-61c151c2-c0e4-43fa-af6e-a2be79f2d7b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:10:01.865488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:01.866748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:01.868038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:10:01.868038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:10:01.868038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:01.868038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:01.869858 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 623/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:01.875685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe62c9-97de-4b46-9cf5-f597fef728d4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:01.981007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:01.990005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe62c9-97de-4b46-9cf5-f597fef728d4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:02.031875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec903e91-8103-4036-82b2-d7170b0ddeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:02.032846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec903e91-8103-4036-82b2-d7170b0ddeeb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:02.033087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec903e91-8103-4036-82b2-d7170b0ddeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:02.033355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec903e91-8103-4036-82b2-d7170b0ddeeb None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:02.034260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec903e91-8103-4036-82b2-d7170b0ddeeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:02.034565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec903e91-8103-4036-82b2-d7170b0ddeeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:02.036006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec903e91-8103-4036-82b2-d7170b0ddeeb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:02.048499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 620/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:10:02.054569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c151c2-c0e4-43fa-af6e-a2be79f2d7b4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:02.145152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:02.168999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c151c2-c0e4-43fa-af6e-a2be79f2d7b4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:02.213438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389af82b-5e1d-4bbf-a798-54c4a46d8d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:02.214364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389af82b-5e1d-4bbf-a798-54c4a46d8d28 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:02.214656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389af82b-5e1d-4bbf-a798-54c4a46d8d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:02.215008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389af82b-5e1d-4bbf-a798-54c4a46d8d28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:02.224036 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389af82b-5e1d-4bbf-a798-54c4a46d8d28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:02.225627 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389af82b-5e1d-4bbf-a798-54c4a46d8d28 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-389af82b-5e1d-4bbf-a798-54c4a46d8d28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:10:02.242578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:02.243831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:02.245585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:10:02.245585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:10:02.245585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:10:02.245585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:10:02.245585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:10:02.245585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:10:02.245585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:10:02.245585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:02.245585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 624/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:10:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:02.251721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec903e91-8103-4036-82b2-d7170b0ddeeb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:02.385567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec903e91-8103-4036-82b2-d7170b0ddeeb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:02.426088 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355249e-0a54-4665-a067-cca56a14c8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:02.426912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355249e-0a54-4665-a067-cca56a14c8cb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:02.427257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355249e-0a54-4665-a067-cca56a14c8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:02.427658 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355249e-0a54-4665-a067-cca56a14c8cb None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:02.428566 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355249e-0a54-4665-a067-cca56a14c8cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:02.429473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355249e-0a54-4665-a067-cca56a14c8cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:02.430498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355249e-0a54-4665-a067-cca56a14c8cb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:02.451920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 621/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:10:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:02.461449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389af82b-5e1d-4bbf-a798-54c4a46d8d28 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:02.581000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389af82b-5e1d-4bbf-a798-54c4a46d8d28 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:02.620990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4a860-5670-4a4d-98b8-74b26055a155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:02.621659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4a860-5670-4a4d-98b8-74b26055a155 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:02.621981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4a860-5670-4a4d-98b8-74b26055a155 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:02.622230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4a860-5670-4a4d-98b8-74b26055a155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:02.623074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4a860-5670-4a4d-98b8-74b26055a155 None admin] RBAC: Adding query filter params (role.id=6bc01dbf83c844fe802466d3c4ada6af, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=6719421144734ef49b3809df30a45961) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:02.623399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4a860-5670-4a4d-98b8-74b26055a155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:02.624892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4a860-5670-4a4d-98b8-74b26055a155 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:02.633239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 625/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:10:02 2022] GET /identity/v3/role_assignments?user.id=6719421144734ef49b3809df30a45961&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=6bc01dbf83c844fe802466d3c4ada6af => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:05.009066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d51de0-4843-4cf0-b2cc-86a264848d34 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:05.009638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d51de0-4843-4cf0-b2cc-86a264848d34 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:05.009638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d51de0-4843-4cf0-b2cc-86a264848d34 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:05.011219 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 622/1247] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:10:05.017521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b47821-d1ac-4ad9-aa2f-391ae769f8b6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:05.018045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b47821-d1ac-4ad9-aa2f-391ae769f8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:05.018153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b47821-d1ac-4ad9-aa2f-391ae769f8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:05.023933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:05.037628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b0b47821-d1ac-4ad9-aa2f-391ae769f8b6 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:05.038921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b0b47821-d1ac-4ad9-aa2f-391ae769f8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:05.118038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 626/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:10:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:10:05.145754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d51de0-4843-4cf0-b2cc-86a264848d34 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:05.166611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:05.277146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d51de0-4843-4cf0-b2cc-86a264848d34 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:05.318610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9012e4-106a-42bf-ab6f-d554456099d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:05.319418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9012e4-106a-42bf-ab6f-d554456099d0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:05.319700 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9012e4-106a-42bf-ab6f-d554456099d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:05.319988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9012e4-106a-42bf-ab6f-d554456099d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:05.332110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9012e4-106a-42bf-ab6f-d554456099d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0f9012e4-106a-42bf-ab6f-d554456099d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:10:05.339746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:05.340771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:05.341628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 623/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:10:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:05.347310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b47821-d1ac-4ad9-aa2f-391ae769f8b6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:05.463998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b47821-d1ac-4ad9-aa2f-391ae769f8b6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:05.498257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:05.505554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c911be-0eaa-4973-9dc7-96efc86c6545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:05.506323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c911be-0eaa-4973-9dc7-96efc86c6545 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:05.506632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c911be-0eaa-4973-9dc7-96efc86c6545 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:05.506932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c911be-0eaa-4973-9dc7-96efc86c6545 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:05.507799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c911be-0eaa-4973-9dc7-96efc86c6545 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:05.508156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c911be-0eaa-4973-9dc7-96efc86c6545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:05.509688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c911be-0eaa-4973-9dc7-96efc86c6545 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:05.519880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 627/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:10:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:05.526236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9012e4-106a-42bf-ab6f-d554456099d0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:05.635765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:05.668515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9012e4-106a-42bf-ab6f-d554456099d0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:05.706627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b54a26-b46a-42a5-ac6f-eae4ea2a1e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:05.707438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b54a26-b46a-42a5-ac6f-eae4ea2a1e4a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:05.707716 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b54a26-b46a-42a5-ac6f-eae4ea2a1e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:05.708008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b54a26-b46a-42a5-ac6f-eae4ea2a1e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:05.719456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b54a26-b46a-42a5-ac6f-eae4ea2a1e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:05.720923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b54a26-b46a-42a5-ac6f-eae4ea2a1e4a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a9b54a26-b46a-42a5-ac6f-eae4ea2a1e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:10:05.739495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:05.740620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:05.741606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:10:05.741606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:10:05.741606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:05.741606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:05.741606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 624/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:05.746884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c911be-0eaa-4973-9dc7-96efc86c6545 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:05.864410 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c911be-0eaa-4973-9dc7-96efc86c6545 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:05.902734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053519f-07ba-43e0-8372-00c6db37e4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:05.903353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053519f-07ba-43e0-8372-00c6db37e4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:05.903652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053519f-07ba-43e0-8372-00c6db37e4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:05.903938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053519f-07ba-43e0-8372-00c6db37e4c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:05.904630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053519f-07ba-43e0-8372-00c6db37e4c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:05.904935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053519f-07ba-43e0-8372-00c6db37e4c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:05.906338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053519f-07ba-43e0-8372-00c6db37e4c2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:05.919622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 628/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:05.925312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b54a26-b46a-42a5-ac6f-eae4ea2a1e4a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:06.040566 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b54a26-b46a-42a5-ac6f-eae4ea2a1e4a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:06.067915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:06.076685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3895e-e56f-4bfe-97e1-9daa00e85c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:06.077565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3895e-e56f-4bfe-97e1-9daa00e85c92 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:06.077900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3895e-e56f-4bfe-97e1-9daa00e85c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:06.078158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3895e-e56f-4bfe-97e1-9daa00e85c92 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:06.087070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3895e-e56f-4bfe-97e1-9daa00e85c92 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:06.088583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3895e-e56f-4bfe-97e1-9daa00e85c92 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-fad3895e-e56f-4bfe-97e1-9daa00e85c92 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:10:06.105448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:06.106676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:06.107745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:06.107745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:06.107745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:10:06.107745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:10:06.107745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:10:06.107745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:10:06.107745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:10:06.107745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:10:06.107745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 16:10:06.107745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:06.107745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 625/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:06.115741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053519f-07ba-43e0-8372-00c6db37e4c2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:06.179457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:06.237369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053519f-07ba-43e0-8372-00c6db37e4c2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:06.277640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d4c08-0afb-4df5-ae87-ceed548ae417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:06.278654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d4c08-0afb-4df5-ae87-ceed548ae417 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:06.278943 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d4c08-0afb-4df5-ae87-ceed548ae417 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:06.279234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d4c08-0afb-4df5-ae87-ceed548ae417 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:06.280330 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d4c08-0afb-4df5-ae87-ceed548ae417 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:06.280846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d4c08-0afb-4df5-ae87-ceed548ae417 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:06.283655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d4c08-0afb-4df5-ae87-ceed548ae417 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:06.305399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 629/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Wed May 25 16:10:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:06.312423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3895e-e56f-4bfe-97e1-9daa00e85c92 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:06.428273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3895e-e56f-4bfe-97e1-9daa00e85c92 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:06.468126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e8623-df2b-41f1-99ec-1439a26bda0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:06.468790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e8623-df2b-41f1-99ec-1439a26bda0a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:06.469171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e8623-df2b-41f1-99ec-1439a26bda0a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:06.469341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e8623-df2b-41f1-99ec-1439a26bda0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:06.530708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:06.544401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e8623-df2b-41f1-99ec-1439a26bda0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:06.545888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e8623-df2b-41f1-99ec-1439a26bda0a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:06.656519 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 626/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:10:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:10:06.663374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d4c08-0afb-4df5-ae87-ceed548ae417 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:06.776307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d4c08-0afb-4df5-ae87-ceed548ae417 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:06.792777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:06.817627 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0169-b495-4451-acd3-5e82258690d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:06.818574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0169-b495-4451-acd3-5e82258690d9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:06.818914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0169-b495-4451-acd3-5e82258690d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:06.819186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0169-b495-4451-acd3-5e82258690d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:06.832381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f0169-b495-4451-acd3-5e82258690d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:06.834120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f0169-b495-4451-acd3-5e82258690d9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f84f0169-b495-4451-acd3-5e82258690d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:10:06.854302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:06.855442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:06.856515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:10:06.856515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:10:06.856515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:06.856515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:06.856515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 630/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:06.862236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e8623-df2b-41f1-99ec-1439a26bda0a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:06.965272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:06.980224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e8623-df2b-41f1-99ec-1439a26bda0a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:07.019122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b2b8d-082b-49d8-b2db-cb7d68f2c066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:07.019793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b2b8d-082b-49d8-b2db-cb7d68f2c066 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:07.020074 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b2b8d-082b-49d8-b2db-cb7d68f2c066 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:07.020441 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b2b8d-082b-49d8-b2db-cb7d68f2c066 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:07.021160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b2b8d-082b-49d8-b2db-cb7d68f2c066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:07.021456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b2b8d-082b-49d8-b2db-cb7d68f2c066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:07.022863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b2b8d-082b-49d8-b2db-cb7d68f2c066 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:07.036253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 627/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:07.042378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0169-b495-4451-acd3-5e82258690d9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:07.159140 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0169-b495-4451-acd3-5e82258690d9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:07.209178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25db72-1dbb-4f52-ba6d-afa9ba4d5fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:07.210282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25db72-1dbb-4f52-ba6d-afa9ba4d5fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:07.210725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25db72-1dbb-4f52-ba6d-afa9ba4d5fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:07.211120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25db72-1dbb-4f52-ba6d-afa9ba4d5fb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:07.220277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25db72-1dbb-4f52-ba6d-afa9ba4d5fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:07.222014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25db72-1dbb-4f52-ba6d-afa9ba4d5fb1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5d25db72-1dbb-4f52-ba6d-afa9ba4d5fb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:10:07.241003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:07.242308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:07.243199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:10:07.243199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:10:07.243199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:10:07.243199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:10:07.243199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:10:07.243199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:10:07.243199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:10:07.243199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:07.243199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 631/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:10:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:07.248654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b2b8d-082b-49d8-b2db-cb7d68f2c066 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:07.367218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b2b8d-082b-49d8-b2db-cb7d68f2c066 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:07.404586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c94801-acf4-45e8-b4ee-991ca564c605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:07.405255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c94801-acf4-45e8-b4ee-991ca564c605 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:07.405541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c94801-acf4-45e8-b4ee-991ca564c605 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:07.405852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c94801-acf4-45e8-b4ee-991ca564c605 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:07.406647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c94801-acf4-45e8-b4ee-991ca564c605 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:07.406956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c94801-acf4-45e8-b4ee-991ca564c605 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:07.408671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c94801-acf4-45e8-b4ee-991ca564c605 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:07.422512 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 628/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:10:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:07.429295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25db72-1dbb-4f52-ba6d-afa9ba4d5fb1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:07.462253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:07.549257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25db72-1dbb-4f52-ba6d-afa9ba4d5fb1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:07.587799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09dba5-27cb-426f-9b51-d7342465f2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:07.588468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09dba5-27cb-426f-9b51-d7342465f2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:07.588752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09dba5-27cb-426f-9b51-d7342465f2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:07.589029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09dba5-27cb-426f-9b51-d7342465f2b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:07.589904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09dba5-27cb-426f-9b51-d7342465f2b6 None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=6719421144734ef49b3809df30a45961) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:07.590218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09dba5-27cb-426f-9b51-d7342465f2b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:07.591678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09dba5-27cb-426f-9b51-d7342465f2b6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:07.598370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 632/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:10:07 2022] GET /identity/v3/role_assignments?user.id=6719421144734ef49b3809df30a45961&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:09.811707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b494-de1a-444f-a177-fba2512821d8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:09.812256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b494-de1a-444f-a177-fba2512821d8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:09.812256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b494-de1a-444f-a177-fba2512821d8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:09.813624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 629/1261] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:10:09.819085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deba53e-863a-4d4b-accf-9d9e977d8d15 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:09.819485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deba53e-863a-4d4b-accf-9d9e977d8d15 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:09.819639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deba53e-863a-4d4b-accf-9d9e977d8d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:09.836737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9deba53e-863a-4d4b-accf-9d9e977d8d15 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:09.837945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9deba53e-863a-4d4b-accf-9d9e977d8d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:09.902235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 633/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:10:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:10:09.925702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b494-de1a-444f-a177-fba2512821d8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:09.936427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:10.027742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b494-de1a-444f-a177-fba2512821d8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:10.061355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc77b96-0df3-4f79-8cd6-c8ad3ffe72d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:10.062218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc77b96-0df3-4f79-8cd6-c8ad3ffe72d6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:10.062443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc77b96-0df3-4f79-8cd6-c8ad3ffe72d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:10.062712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc77b96-0df3-4f79-8cd6-c8ad3ffe72d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:10.072769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc77b96-0df3-4f79-8cd6-c8ad3ffe72d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6fc77b96-0df3-4f79-8cd6-c8ad3ffe72d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:10:10.080238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:10.081954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:10.082971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 630/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:10:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:10.087968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deba53e-863a-4d4b-accf-9d9e977d8d15 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:10.106827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:10.193063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deba53e-863a-4d4b-accf-9d9e977d8d15 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:10.239564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153c28-f87d-4e13-b015-bfaa003ac76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:10.240428 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153c28-f87d-4e13-b015-bfaa003ac76a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:10.240783 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153c28-f87d-4e13-b015-bfaa003ac76a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:10.241174 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153c28-f87d-4e13-b015-bfaa003ac76a None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:10.242259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2153c28-f87d-4e13-b015-bfaa003ac76a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:10.242889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2153c28-f87d-4e13-b015-bfaa003ac76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:10.244803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2153c28-f87d-4e13-b015-bfaa003ac76a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:10.255047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 634/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:10:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:10.260901 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc77b96-0df3-4f79-8cd6-c8ad3ffe72d6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:10.327140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:10.360760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc77b96-0df3-4f79-8cd6-c8ad3ffe72d6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:10.399170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca42f8-a18e-45e2-b775-67dbc83d4429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:10.399967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca42f8-a18e-45e2-b775-67dbc83d4429 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:10.400254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca42f8-a18e-45e2-b775-67dbc83d4429 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:10.400533 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca42f8-a18e-45e2-b775-67dbc83d4429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:10.411477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca42f8-a18e-45e2-b775-67dbc83d4429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:10.412923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca42f8-a18e-45e2-b775-67dbc83d4429 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c0ca42f8-a18e-45e2-b775-67dbc83d4429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:10:10.429848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:10.430791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:10.431697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:10:10.431697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:10:10.431697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:10.431697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:10.431697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 631/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:10.436783 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153c28-f87d-4e13-b015-bfaa003ac76a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:10.518001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:10.537156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153c28-f87d-4e13-b015-bfaa003ac76a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:10.570615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815f9f0-298e-43e1-bd12-1c7b48452030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:10.571276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815f9f0-298e-43e1-bd12-1c7b48452030 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:10.571561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815f9f0-298e-43e1-bd12-1c7b48452030 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:10.571838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815f9f0-298e-43e1-bd12-1c7b48452030 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:10.572561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815f9f0-298e-43e1-bd12-1c7b48452030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:10.572875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815f9f0-298e-43e1-bd12-1c7b48452030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:10.574313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815f9f0-298e-43e1-bd12-1c7b48452030 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:10.586367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 635/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:10.591994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca42f8-a18e-45e2-b775-67dbc83d4429 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:10.692031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca42f8-a18e-45e2-b775-67dbc83d4429 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:10.710986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:10.725233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c4928-7d18-4ce4-9b69-8c87cfa4f27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:10.726070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c4928-7d18-4ce4-9b69-8c87cfa4f27c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:10.726356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c4928-7d18-4ce4-9b69-8c87cfa4f27c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:10.726634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c4928-7d18-4ce4-9b69-8c87cfa4f27c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:10.733976 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c4928-7d18-4ce4-9b69-8c87cfa4f27c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:10.735408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c4928-7d18-4ce4-9b69-8c87cfa4f27c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-091c4928-7d18-4ce4-9b69-8c87cfa4f27c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:10:10.752401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:10.754310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:10.755931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:10.755931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:10.755931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:10:10.755931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:10:10.755931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:10:10.755931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:10:10.755931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:10:10.755931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:10:10.755931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 16:10:10.755931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:10.755931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 632/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:10.760175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815f9f0-298e-43e1-bd12-1c7b48452030 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:10.857604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815f9f0-298e-43e1-bd12-1c7b48452030 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:10.891179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe91395-58ee-410f-b2a6-a162d61ba4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:10.891831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe91395-58ee-410f-b2a6-a162d61ba4ac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:10.892114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe91395-58ee-410f-b2a6-a162d61ba4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:10.892389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe91395-58ee-410f-b2a6-a162d61ba4ac None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:10.893218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe91395-58ee-410f-b2a6-a162d61ba4ac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:10.893525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe91395-58ee-410f-b2a6-a162d61ba4ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:10.895154 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe91395-58ee-410f-b2a6-a162d61ba4ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:10.903659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:10.912954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 636/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Wed May 25 16:10:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:10.918719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c4928-7d18-4ce4-9b69-8c87cfa4f27c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:11.017941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c4928-7d18-4ce4-9b69-8c87cfa4f27c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:11.051933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b75cc-da2e-4ce9-80bb-2eb8d598fb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:11.052716 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b75cc-da2e-4ce9-80bb-2eb8d598fb8a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:11.052996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b75cc-da2e-4ce9-80bb-2eb8d598fb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:11.053265 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b75cc-da2e-4ce9-80bb-2eb8d598fb8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:11.063726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b75cc-da2e-4ce9-80bb-2eb8d598fb8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:11.065154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b75cc-da2e-4ce9-80bb-2eb8d598fb8a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-298b75cc-da2e-4ce9-80bb-2eb8d598fb8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:10:11.082632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:11.083758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:11.084857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:10:11.084857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:10:11.084857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:11.084857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:11.084857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 633/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:11.089506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe91395-58ee-410f-b2a6-a162d61ba4ac None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:11.190047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe91395-58ee-410f-b2a6-a162d61ba4ac None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:11.227546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67938f02-ceac-4293-8c9c-fb77c6e630b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:11.228206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67938f02-ceac-4293-8c9c-fb77c6e630b5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:11.228495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67938f02-ceac-4293-8c9c-fb77c6e630b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:11.228771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67938f02-ceac-4293-8c9c-fb77c6e630b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:11.229490 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67938f02-ceac-4293-8c9c-fb77c6e630b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:11.229825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67938f02-ceac-4293-8c9c-fb77c6e630b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:11.231199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67938f02-ceac-4293-8c9c-fb77c6e630b5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:11.243035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 637/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:11.248694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b75cc-da2e-4ce9-80bb-2eb8d598fb8a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:11.280501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:11.348955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b75cc-da2e-4ce9-80bb-2eb8d598fb8a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:11.381453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa097c6-d584-439f-94bf-753c9c1d80e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:11.382266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa097c6-d584-439f-94bf-753c9c1d80e3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:11.382551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa097c6-d584-439f-94bf-753c9c1d80e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:11.382824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa097c6-d584-439f-94bf-753c9c1d80e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:11.389412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa097c6-d584-439f-94bf-753c9c1d80e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:11.390878 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa097c6-d584-439f-94bf-753c9c1d80e3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1aa097c6-d584-439f-94bf-753c9c1d80e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:10:11.405408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:11.406495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:11.407697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:10:11.407697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:10:11.407697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:10:11.407697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:10:11.407697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:10:11.407697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:10:11.407697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:10:11.407697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:11.407697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 634/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:10:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:11.413346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67938f02-ceac-4293-8c9c-fb77c6e630b5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:11.471040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:11.519506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67938f02-ceac-4293-8c9c-fb77c6e630b5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:11.554929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118052f9-2886-41f0-88ba-bc6be494321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:11.555575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118052f9-2886-41f0-88ba-bc6be494321c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:11.555853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118052f9-2886-41f0-88ba-bc6be494321c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:11.556124 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118052f9-2886-41f0-88ba-bc6be494321c None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:11.556920 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118052f9-2886-41f0-88ba-bc6be494321c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:11.557225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118052f9-2886-41f0-88ba-bc6be494321c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:11.558683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118052f9-2886-41f0-88ba-bc6be494321c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:11.571309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 638/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:10:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:11.577424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa097c6-d584-439f-94bf-753c9c1d80e3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:11.667483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:11.676471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa097c6-d584-439f-94bf-753c9c1d80e3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:11.710068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d9bc8-bc6a-4a70-a440-fbd872c16f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:11.710969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d9bc8-bc6a-4a70-a440-fbd872c16f8b None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:11.711251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d9bc8-bc6a-4a70-a440-fbd872c16f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:11.711527 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d9bc8-bc6a-4a70-a440-fbd872c16f8b None admin] PATH_INFO: `/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/6719421144734ef49b3809df30a45961/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:11.713448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d9bc8-bc6a-4a70-a440-fbd872c16f8b None admin] RBAC: Authorizing `identity:create_grant(project_id=27d1825035a24259a01beb6b6610ab3f, user_id=6719421144734ef49b3809df30a45961, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:11.714904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d9bc8-bc6a-4a70-a440-fbd872c16f8b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:11.728447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 635/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Wed May 25 16:10:11 2022] PUT /identity/v3/projects/27d1825035a24259a01beb6b6610ab3f/users/6719421144734ef49b3809df30a45961/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:10:13.956725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72b7d7-7a16-4e00-ae0c-334d993dafc6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:13.957477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72b7d7-7a16-4e00-ae0c-334d993dafc6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:13.957477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72b7d7-7a16-4e00-ae0c-334d993dafc6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:13.958695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 639/1274] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:10:13.963955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe298ab-d878-4111-b95a-e57e3bbe0aae None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:13.964368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe298ab-d878-4111-b95a-e57e3bbe0aae None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:13.964511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe298ab-d878-4111-b95a-e57e3bbe0aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:13.981433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8fe298ab-d878-4111-b95a-e57e3bbe0aae None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:13.982651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8fe298ab-d878-4111-b95a-e57e3bbe0aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:14.048559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 636/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:10:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:10:14.072977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72b7d7-7a16-4e00-ae0c-334d993dafc6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:14.173266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72b7d7-7a16-4e00-ae0c-334d993dafc6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:14.194570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:14.217664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0740c-060f-4874-8716-772c34664312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:14.218482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0740c-060f-4874-8716-772c34664312 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:14.218808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0740c-060f-4874-8716-772c34664312 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:14.219119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0740c-060f-4874-8716-772c34664312 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:14.231412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0740c-060f-4874-8716-772c34664312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-aed0740c-060f-4874-8716-772c34664312 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 16:10:14.239848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:14.241373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:14.242805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 640/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Wed May 25 16:10:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:14.247473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe298ab-d878-4111-b95a-e57e3bbe0aae None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:14.327733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:14.347093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe298ab-d878-4111-b95a-e57e3bbe0aae None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:14.386876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3be6e-1b2e-4dbb-8d8d-0cd517eedb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:14.387545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3be6e-1b2e-4dbb-8d8d-0cd517eedb08 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:14.387825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3be6e-1b2e-4dbb-8d8d-0cd517eedb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:14.388096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3be6e-1b2e-4dbb-8d8d-0cd517eedb08 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:14.388836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3be6e-1b2e-4dbb-8d8d-0cd517eedb08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:14.389139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3be6e-1b2e-4dbb-8d8d-0cd517eedb08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:14.390544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3be6e-1b2e-4dbb-8d8d-0cd517eedb08 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:14.400667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 637/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Wed May 25 16:10:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:14.406662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0740c-060f-4874-8716-772c34664312 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:14.508438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0740c-060f-4874-8716-772c34664312 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:14.544030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d11e9-dbc8-4c92-9aba-208c40b06fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:14.544818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d11e9-dbc8-4c92-9aba-208c40b06fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:14.545095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d11e9-dbc8-4c92-9aba-208c40b06fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:14.545367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d11e9-dbc8-4c92-9aba-208c40b06fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:14.560795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d11e9-dbc8-4c92-9aba-208c40b06fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:14.562304 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d11e9-dbc8-4c92-9aba-208c40b06fd8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-eb9d11e9-dbc8-4c92-9aba-208c40b06fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:10:14.579724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:14.581105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:14.582671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:10:14.582671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:10:14.582671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:14.582671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:14.582671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 641/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:14.587269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3be6e-1b2e-4dbb-8d8d-0cd517eedb08 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:14.698536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3be6e-1b2e-4dbb-8d8d-0cd517eedb08 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:14.731348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90407b82-9d90-48d0-bf98-3a28241d5aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:14.732001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90407b82-9d90-48d0-bf98-3a28241d5aad None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:14.732279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90407b82-9d90-48d0-bf98-3a28241d5aad None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:14.732548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90407b82-9d90-48d0-bf98-3a28241d5aad None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:14.733284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90407b82-9d90-48d0-bf98-3a28241d5aad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:14.733585 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90407b82-9d90-48d0-bf98-3a28241d5aad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:14.735010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90407b82-9d90-48d0-bf98-3a28241d5aad None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:14.746609 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 638/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:14.752257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d11e9-dbc8-4c92-9aba-208c40b06fd8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:14.783723 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:14.855766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d11e9-dbc8-4c92-9aba-208c40b06fd8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:14.888601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84dcc4-a866-423b-bba5-0aa915cbff34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:14.889399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84dcc4-a866-423b-bba5-0aa915cbff34 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:14.889688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84dcc4-a866-423b-bba5-0aa915cbff34 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:14.890003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84dcc4-a866-423b-bba5-0aa915cbff34 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:14.898059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84dcc4-a866-423b-bba5-0aa915cbff34 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:14.899505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84dcc4-a866-423b-bba5-0aa915cbff34 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7e84dcc4-a866-423b-bba5-0aa915cbff34 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 16:10:14.914025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:14.915147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:14.916177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:14.916177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:14.916177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 16:10:14.916177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 16:10:14.916177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 16:10:14.916177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 16:10:14.916177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 16:10:14.916177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 16:10:14.916177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 16:10:14.916177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:14.916177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 642/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:14.921031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90407b82-9d90-48d0-bf98-3a28241d5aad None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:14.924920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:15.027060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90407b82-9d90-48d0-bf98-3a28241d5aad None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:15.060553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8481412-35b2-4b5f-8fd1-ddda0de548de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:15.061222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8481412-35b2-4b5f-8fd1-ddda0de548de None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:15.061508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8481412-35b2-4b5f-8fd1-ddda0de548de None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:15.061814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8481412-35b2-4b5f-8fd1-ddda0de548de None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:15.062639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8481412-35b2-4b5f-8fd1-ddda0de548de None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:15.062951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8481412-35b2-4b5f-8fd1-ddda0de548de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:15.064388 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8481412-35b2-4b5f-8fd1-ddda0de548de None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:15.082492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 639/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Wed May 25 16:10:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:15.088538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84dcc4-a866-423b-bba5-0aa915cbff34 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:15.178668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:15.192586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84dcc4-a866-423b-bba5-0aa915cbff34 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:15.225999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c1f30-ade3-4da9-b852-8197a01eda5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:15.226639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c1f30-ade3-4da9-b852-8197a01eda5c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:15.226918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c1f30-ade3-4da9-b852-8197a01eda5c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:15.227191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c1f30-ade3-4da9-b852-8197a01eda5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:15.295418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c1f30-ade3-4da9-b852-8197a01eda5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:15.296865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c1f30-ade3-4da9-b852-8197a01eda5c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:15.384566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:15.395783 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 643/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Wed May 25 16:10:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 16:10:15.401894 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8481412-35b2-4b5f-8fd1-ddda0de548de None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:15.464264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:15.502907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8481412-35b2-4b5f-8fd1-ddda0de548de None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:15.536734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc4f11-aa26-4950-9e0c-5e9aeb31d119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:15.537521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc4f11-aa26-4950-9e0c-5e9aeb31d119 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:15.537826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc4f11-aa26-4950-9e0c-5e9aeb31d119 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:15.538101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc4f11-aa26-4950-9e0c-5e9aeb31d119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:15.547670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc4f11-aa26-4950-9e0c-5e9aeb31d119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:15.549097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc4f11-aa26-4950-9e0c-5e9aeb31d119 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-81cc4f11-aa26-4950-9e0c-5e9aeb31d119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 16:10:15.567573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:15.568707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:15.569836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 16:10:15.569836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 16:10:15.569836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 16:10:15.569836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:15.569836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 640/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:15.574451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c1f30-ade3-4da9-b852-8197a01eda5c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:15.673146 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c1f30-ade3-4da9-b852-8197a01eda5c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:15.708501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aba417-0b2e-4dc6-bebf-f55685161d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:15.709153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aba417-0b2e-4dc6-bebf-f55685161d6e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:15.709436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aba417-0b2e-4dc6-bebf-f55685161d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:15.709741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aba417-0b2e-4dc6-bebf-f55685161d6e None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:15.710457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aba417-0b2e-4dc6-bebf-f55685161d6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:15.710764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aba417-0b2e-4dc6-bebf-f55685161d6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:15.712172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aba417-0b2e-4dc6-bebf-f55685161d6e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:15.723499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 644/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Wed May 25 16:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:15.729110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc4f11-aa26-4950-9e0c-5e9aeb31d119 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:15.828322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc4f11-aa26-4950-9e0c-5e9aeb31d119 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:15.842160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:15.862122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699929c-a0d6-4c99-a1e2-76a8f0cc74da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:15.862894 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699929c-a0d6-4c99-a1e2-76a8f0cc74da None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:15.863170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699929c-a0d6-4c99-a1e2-76a8f0cc74da None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:15.864136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699929c-a0d6-4c99-a1e2-76a8f0cc74da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:15.870682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699929c-a0d6-4c99-a1e2-76a8f0cc74da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:15.872089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699929c-a0d6-4c99-a1e2-76a8f0cc74da None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e699929c-a0d6-4c99-a1e2-76a8f0cc74da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 16:10:15.886331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:15.887373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:15.888403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 16:10:15.888403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 16:10:15.888403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 16:10:15.888403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 16:10:15.888403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 16:10:15.888403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 16:10:15.888403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 16:10:15.888403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 25 16:10:15.888403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 641/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Wed May 25 16:10:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:15.893326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aba417-0b2e-4dc6-bebf-f55685161d6e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:15.943751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:15.992778 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aba417-0b2e-4dc6-bebf-f55685161d6e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:16.025580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acaccb-4b16-41d0-86e4-3bbb50f1f5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:16.026249 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acaccb-4b16-41d0-86e4-3bbb50f1f5cd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:16.026526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acaccb-4b16-41d0-86e4-3bbb50f1f5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:16.026798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acaccb-4b16-41d0-86e4-3bbb50f1f5cd None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:16.027620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acaccb-4b16-41d0-86e4-3bbb50f1f5cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:16.027877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acaccb-4b16-41d0-86e4-3bbb50f1f5cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:16.029291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acaccb-4b16-41d0-86e4-3bbb50f1f5cd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:16.041663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 645/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Wed May 25 16:10:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:16.048080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699929c-a0d6-4c99-a1e2-76a8f0cc74da None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:16.147046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699929c-a0d6-4c99-a1e2-76a8f0cc74da None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:16.182325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17186ee7-69e2-48b4-b472-275c3a6e9719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:16.183058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186ee7-69e2-48b4-b472-275c3a6e9719 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:16.183383 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186ee7-69e2-48b4-b472-275c3a6e9719 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:16.183696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186ee7-69e2-48b4-b472-275c3a6e9719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:16.184636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17186ee7-69e2-48b4-b472-275c3a6e9719 None admin] RBAC: Adding query filter params (role.id=d38fc08199294a1e9ed590da019eaf86, scope.project.id=27d1825035a24259a01beb6b6610ab3f, user.id=6719421144734ef49b3809df30a45961) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:16.184972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17186ee7-69e2-48b4-b472-275c3a6e9719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:16.186571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17186ee7-69e2-48b4-b472-275c3a6e9719 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:16.192841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 642/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Wed May 25 16:10:16 2022] GET /identity/v3/role_assignments?user.id=6719421144734ef49b3809df30a45961&scope.project.id=27d1825035a24259a01beb6b6610ab3f&role.id=d38fc08199294a1e9ed590da019eaf86 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:18.433988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dcdd3-cd57-4ae3-bd4c-5d81d3824aeb None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:18.434441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dcdd3-cd57-4ae3-bd4c-5d81d3824aeb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:18.434530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dcdd3-cd57-4ae3-bd4c-5d81d3824aeb None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:18.435913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 646/1288] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:10:18.441154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb0bcd-db26-4996-b233-86cade215afe None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:18.441596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb0bcd-db26-4996-b233-86cade215afe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:18.441798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb0bcd-db26-4996-b233-86cade215afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:18.458399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9fdb0bcd-db26-4996-b233-86cade215afe None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:18.459584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9fdb0bcd-db26-4996-b233-86cade215afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:18.489656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:18.533024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 643/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:10:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:10:18.556487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dcdd3-cd57-4ae3-bd4c-5d81d3824aeb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:18.655315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dcdd3-cd57-4ae3-bd4c-5d81d3824aeb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:18.666378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:18.689582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4fca3-5648-460c-9bc3-0441cf346a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:18.690336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4fca3-5648-460c-9bc3-0441cf346a52 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:18.690616 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4fca3-5648-460c-9bc3-0441cf346a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:18.690886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4fca3-5648-460c-9bc3-0441cf346a52 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:18.691613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4fca3-5648-460c-9bc3-0441cf346a52 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:18.692971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4fca3-5648-460c-9bc3-0441cf346a52 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c0d4fca3-5648-460c-9bc3-0441cf346a52 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:18.704468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:18.705629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:18.706571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 647/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Wed May 25 16:10:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:18.712397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb0bcd-db26-4996-b233-86cade215afe None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:18.809776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb0bcd-db26-4996-b233-86cade215afe None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:18.843927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4306b-6494-4b3d-9606-f4ed7cf35d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:18.844587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4306b-6494-4b3d-9606-f4ed7cf35d6d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:18.844869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4306b-6494-4b3d-9606-f4ed7cf35d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:18.845144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4306b-6494-4b3d-9606-f4ed7cf35d6d None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:18.845877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4306b-6494-4b3d-9606-f4ed7cf35d6d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:18.846184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4306b-6494-4b3d-9606-f4ed7cf35d6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:18.848200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4306b-6494-4b3d-9606-f4ed7cf35d6d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:18.853128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 644/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Wed May 25 16:10:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:18.858880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4fca3-5648-460c-9bc3-0441cf346a52 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:18.960335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4fca3-5648-460c-9bc3-0441cf346a52 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:18.994645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361502c1-288b-43ae-90d3-04a0c776058c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:18.995379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361502c1-288b-43ae-90d3-04a0c776058c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:18.995705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361502c1-288b-43ae-90d3-04a0c776058c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:18.996025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361502c1-288b-43ae-90d3-04a0c776058c None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:18.996797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361502c1-288b-43ae-90d3-04a0c776058c None admin] RBAC: Adding query filter params (type=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:18.997161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361502c1-288b-43ae-90d3-04a0c776058c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:18.998624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361502c1-288b-43ae-90d3-04a0c776058c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:19.003857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 648/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Wed May 25 16:10:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:21.212269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fc49f-2d44-44fb-bf4c-bbad0a3685f9 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:21.212947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fc49f-2d44-44fb-bf4c-bbad0a3685f9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:21.212947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fc49f-2d44-44fb-bf4c-bbad0a3685f9 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:21.214521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 645/1293] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:10:21.220335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231029f8-958e-4130-a1b1-6239fc8d2e46 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:21.220821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231029f8-958e-4130-a1b1-6239fc8d2e46 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:21.221010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231029f8-958e-4130-a1b1-6239fc8d2e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:21.240092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-231029f8-958e-4130-a1b1-6239fc8d2e46 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:21.241431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-231029f8-958e-4130-a1b1-6239fc8d2e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:21.272808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:21.309572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 649/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:10:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:10:21.332220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fc49f-2d44-44fb-bf4c-bbad0a3685f9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:21.362700 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:21.435411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fc49f-2d44-44fb-bf4c-bbad0a3685f9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:21.470108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8a1e2-0425-46e4-9f3f-08d172df7b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:21.470698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8a1e2-0425-46e4-9f3f-08d172df7b8d None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:21.470985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8a1e2-0425-46e4-9f3f-08d172df7b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:21.471264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8a1e2-0425-46e4-9f3f-08d172df7b8d None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:21.471981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8a1e2-0425-46e4-9f3f-08d172df7b8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:21.473490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8a1e2-0425-46e4-9f3f-08d172df7b8d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:21.486808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 646/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Wed May 25 16:10:21 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:10:23.750664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9376f2-eab9-4a6a-adf9-4d828e70f8e2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:23.751254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9376f2-eab9-4a6a-adf9-4d828e70f8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:23.751254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9376f2-eab9-4a6a-adf9-4d828e70f8e2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:23.752531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 650/1296] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:10:23.757701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48097c9b-2a97-4abf-8b0d-b35fee7345b4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:23.758139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48097c9b-2a97-4abf-8b0d-b35fee7345b4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:23.758266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48097c9b-2a97-4abf-8b0d-b35fee7345b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:23.774840 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-48097c9b-2a97-4abf-8b0d-b35fee7345b4 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:23.776006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-48097c9b-2a97-4abf-8b0d-b35fee7345b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:23.847037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 647/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:10:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:10:23.875950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9376f2-eab9-4a6a-adf9-4d828e70f8e2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:23.988926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9376f2-eab9-4a6a-adf9-4d828e70f8e2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:24.027444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e45b6-8f33-4780-af7b-35dcbd6322e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:24.028163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e45b6-8f33-4780-af7b-35dcbd6322e3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:24.028446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e45b6-8f33-4780-af7b-35dcbd6322e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:24.028722 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e45b6-8f33-4780-af7b-35dcbd6322e3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:24.029400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e45b6-8f33-4780-af7b-35dcbd6322e3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:24.030799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e45b6-8f33-4780-af7b-35dcbd6322e3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-477e45b6-8f33-4780-af7b-35dcbd6322e3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:24.043406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:24.049456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:24.050801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 651/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Wed May 25 16:10:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:24.051887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48097c9b-2a97-4abf-8b0d-b35fee7345b4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:24.068689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:24.149175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48097c9b-2a97-4abf-8b0d-b35fee7345b4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:24.183717 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4ae9-f890-4a83-95c5-2856d8a317e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:24.184425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4ae9-f890-4a83-95c5-2856d8a317e7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:24.184709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4ae9-f890-4a83-95c5-2856d8a317e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:24.184987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4ae9-f890-4a83-95c5-2856d8a317e7 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:24.185707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4ae9-f890-4a83-95c5-2856d8a317e7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:24.186013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4ae9-f890-4a83-95c5-2856d8a317e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:24.187433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4ae9-f890-4a83-95c5-2856d8a317e7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:24.193560 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 648/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Wed May 25 16:10:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:24.201266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e45b6-8f33-4780-af7b-35dcbd6322e3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:24.246407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:24.319206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e45b6-8f33-4780-af7b-35dcbd6322e3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:24.352857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d5274-c2ba-4574-ac7b-b6ea52db2af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:24.354042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d5274-c2ba-4574-ac7b-b6ea52db2af8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:24.354042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d5274-c2ba-4574-ac7b-b6ea52db2af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:24.354321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d5274-c2ba-4574-ac7b-b6ea52db2af8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:24.354886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d5274-c2ba-4574-ac7b-b6ea52db2af8 None admin] RBAC: Adding query filter params (interface=public, service_id=1825ed5f07f24e52ab7a94d3b8232eaf, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:24.355197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d5274-c2ba-4574-ac7b-b6ea52db2af8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:24.356555 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d5274-c2ba-4574-ac7b-b6ea52db2af8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:24.362213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 652/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Wed May 25 16:10:24 2022] GET /identity/v3/endpoints?service_id=1825ed5f07f24e52ab7a94d3b8232eaf&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:24.368281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4ae9-f890-4a83-95c5-2856d8a317e7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:24.454166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:24.472738 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4ae9-f890-4a83-95c5-2856d8a317e7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:24.505628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a6d26-c0cd-43dc-8723-4426a2ab1226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:24.506313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a6d26-c0cd-43dc-8723-4426a2ab1226 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:24.506596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a6d26-c0cd-43dc-8723-4426a2ab1226 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:24.506872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a6d26-c0cd-43dc-8723-4426a2ab1226 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:24.507540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a6d26-c0cd-43dc-8723-4426a2ab1226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:24.508900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a6d26-c0cd-43dc-8723-4426a2ab1226 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:24.515545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 649/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Wed May 25 16:10:24 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:10:26.733233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e741230-8004-4976-94a0-80a039ba767e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:26.733773 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e741230-8004-4976-94a0-80a039ba767e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:26.733773 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e741230-8004-4976-94a0-80a039ba767e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:26.735203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 653/1302] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:10:26.740586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38a8b-4c42-4b14-908a-804defb1a35b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:26.741014 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38a8b-4c42-4b14-908a-804defb1a35b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:26.741145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38a8b-4c42-4b14-908a-804defb1a35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:26.758345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5fa38a8b-4c42-4b14-908a-804defb1a35b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:26.759538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5fa38a8b-4c42-4b14-908a-804defb1a35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:26.826304 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 650/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:10:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 16:10:26.849632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e741230-8004-4976-94a0-80a039ba767e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:26.950178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:26.958622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e741230-8004-4976-94a0-80a039ba767e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:26.992339 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89ec68-d712-4291-8ed4-ab9a5f046296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:26.993055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89ec68-d712-4291-8ed4-ab9a5f046296 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:26.993334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89ec68-d712-4291-8ed4-ab9a5f046296 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:26.993606 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89ec68-d712-4291-8ed4-ab9a5f046296 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:26.994280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89ec68-d712-4291-8ed4-ab9a5f046296 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:26.995641 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89ec68-d712-4291-8ed4-ab9a5f046296 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-df89ec68-d712-4291-8ed4-ab9a5f046296 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 16:10:27.007825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 16:10:27.009017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 16:10:27.010127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 654/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Wed May 25 16:10:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 16:10:27.015171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa38a8b-4c42-4b14-908a-804defb1a35b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:27.090664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:27.113578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa38a8b-4c42-4b14-908a-804defb1a35b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:27.147439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78880a-03f3-4b65-8c46-b3ebc3dc46f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:27.148101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78880a-03f3-4b65-8c46-b3ebc3dc46f2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:27.148389 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78880a-03f3-4b65-8c46-b3ebc3dc46f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:27.148668 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78880a-03f3-4b65-8c46-b3ebc3dc46f2 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:27.149375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78880a-03f3-4b65-8c46-b3ebc3dc46f2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:27.149684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78880a-03f3-4b65-8c46-b3ebc3dc46f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:27.151082 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78880a-03f3-4b65-8c46-b3ebc3dc46f2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:27.156243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 651/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Wed May 25 16:10:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:27.163186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89ec68-d712-4291-8ed4-ab9a5f046296 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:27.263433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89ec68-d712-4291-8ed4-ab9a5f046296 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:27.297273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6107348-d5e3-474e-b810-55b06967a755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:27.297948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6107348-d5e3-474e-b810-55b06967a755 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:27.298233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6107348-d5e3-474e-b810-55b06967a755 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:27.298508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6107348-d5e3-474e-b810-55b06967a755 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:27.299240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6107348-d5e3-474e-b810-55b06967a755 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:27.300749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6107348-d5e3-474e-b810-55b06967a755 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:27.318705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 655/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Wed May 25 16:10:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:10:37.153752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=30 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:37.314117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:33.336704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18705e65-9510-4ca7-aeba-0c119ff49c3a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:33.337330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18705e65-9510-4ca7-aeba-0c119ff49c3a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:33.337330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18705e65-9510-4ca7-aeba-0c119ff49c3a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:33.338976 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 652/1307] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:11:33.344250 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbb4ab-166a-49a5-92af-347f61cf8b56 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:33.345216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbb4ab-166a-49a5-92af-347f61cf8b56 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:33.345502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbb4ab-166a-49a5-92af-347f61cf8b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:33.507548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c3bbb4ab-166a-49a5-92af-347f61cf8b56 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:33.511183 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3bbb4ab-166a-49a5-92af-347f61cf8b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:33.655987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 656/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:33.699947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf7182-3e8b-461b-bc41-3db7d42d45fe None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:33.700390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf7182-3e8b-461b-bc41-3db7d42d45fe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:33.700674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf7182-3e8b-461b-bc41-3db7d42d45fe None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:33.702163 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 653/1309] 10.222.0.42 () {54 vars in 964 bytes} [Wed May 25 16:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:11:33.709311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddc693-ef41-4cdf-90d7-b1d990bc68a2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:33.709679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddc693-ef41-4cdf-90d7-b1d990bc68a2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:33.709988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddc693-ef41-4cdf-90d7-b1d990bc68a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:33.796801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-77ddc693-ef41-4cdf-90d7-b1d990bc68a2 None None] MFA Rules not processed for user `e42c796dbd7441289a7cbc328abeaf81`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:33.798774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-77ddc693-ef41-4cdf-90d7-b1d990bc68a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:33.824173 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:33.896917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 657/1310] 10.222.0.42 () {58 vars in 1061 bytes} [Wed May 25 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:33.909222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf7182-3e8b-461b-bc41-3db7d42d45fe None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:33.911237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2ddf7182-3e8b-461b-bc41-3db7d42d45fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:34.055846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf7182-3e8b-461b-bc41-3db7d42d45fe None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:34.144782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2289c-ea3d-41f0-9d7e-1bca3adc27a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:34.147909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2289c-ea3d-41f0-9d7e-1bca3adc27a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:34.147909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2289c-ea3d-41f0-9d7e-1bca3adc27a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:34.148194 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2289c-ea3d-41f0-9d7e-1bca3adc27a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:34.150174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2be2289c-ea3d-41f0-9d7e-1bca3adc27a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:34.215758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:34.277758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2289c-ea3d-41f0-9d7e-1bca3adc27a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:34.279195 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2289c-ea3d-41f0-9d7e-1bca3adc27a1 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:34.299429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 654/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:11:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:11:37.036769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0d229-2925-4fb5-b7a8-774f7566e3a4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:37.037350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0d229-2925-4fb5-b7a8-774f7566e3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:37.037350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0d229-2925-4fb5-b7a8-774f7566e3a4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:37.038664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 658/1312] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:11:37.043926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a454b10-c755-4668-ae7e-3dd131b846be None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:37.044369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a454b10-c755-4668-ae7e-3dd131b846be None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:37.044565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a454b10-c755-4668-ae7e-3dd131b846be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:37.137284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4a454b10-c755-4668-ae7e-3dd131b846be None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:37.138798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4a454b10-c755-4668-ae7e-3dd131b846be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:37.252583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 655/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:37.291312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0d229-2925-4fb5-b7a8-774f7566e3a4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:37.292763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0fc0d229-2925-4fb5-b7a8-774f7566e3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:37.405436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0d229-2925-4fb5-b7a8-774f7566e3a4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:37.488450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:37.491287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d821f-6d23-43a5-9080-84123a02005c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:37.492177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d821f-6d23-43a5-9080-84123a02005c service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:37.492618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d821f-6d23-43a5-9080-84123a02005c service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:37.493122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d821f-6d23-43a5-9080-84123a02005c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:37.494819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f66d821f-6d23-43a5-9080-84123a02005c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:37.623906 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d821f-6d23-43a5-9080-84123a02005c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:37.625380 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d821f-6d23-43a5-9080-84123a02005c service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:37.646963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 659/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:11:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:11:40.431736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af2c9e-83db-49e3-9a23-657a379150f1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:40.432317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af2c9e-83db-49e3-9a23-657a379150f1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:40.432317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af2c9e-83db-49e3-9a23-657a379150f1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:40.434375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 656/1315] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:11:40.439806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9bdea-c8e4-4d84-8b63-e606bc5a0e20 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:40.440255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9bdea-c8e4-4d84-8b63-e606bc5a0e20 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:40.440353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9bdea-c8e4-4d84-8b63-e606bc5a0e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:40.509246 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e4b9bdea-c8e4-4d84-8b63-e606bc5a0e20 None None] MFA Rules not processed for user `ab37fe904ac74e65a450dec01100babf`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:40.510522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e4b9bdea-c8e4-4d84-8b63-e606bc5a0e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:40.563162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:40.613137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 660/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:11:40 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:40.649971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af2c9e-83db-49e3-9a23-657a379150f1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:40.651586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-58af2c9e-83db-49e3-9a23-657a379150f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:40.661989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:40.757773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af2c9e-83db-49e3-9a23-657a379150f1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:40.834728 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d5493-b771-43cb-82ac-984a455fe192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:40.835383 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d5493-b771-43cb-82ac-984a455fe192 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:40.835662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d5493-b771-43cb-82ac-984a455fe192 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:40.835930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d5493-b771-43cb-82ac-984a455fe192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:40.838910 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ab2d5493-b771-43cb-82ac-984a455fe192 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:40.913709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:40.973288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d5493-b771-43cb-82ac-984a455fe192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:40.974794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d5493-b771-43cb-82ac-984a455fe192 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:40.995655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 657/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:11:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:11:43.831995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db157199-748d-4a03-93c1-5c68553e3df8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:43.832530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db157199-748d-4a03-93c1-5c68553e3df8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:43.832656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db157199-748d-4a03-93c1-5c68553e3df8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:43.834082 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 661/1318] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:11:43.839568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2677fa-7a50-468b-97ea-04d1d4ce462e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:43.839978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2677fa-7a50-468b-97ea-04d1d4ce462e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:43.840137 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2677fa-7a50-468b-97ea-04d1d4ce462e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:43.903664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2a2677fa-7a50-468b-97ea-04d1d4ce462e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:43.904921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2a2677fa-7a50-468b-97ea-04d1d4ce462e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:44.000624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 658/1319] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:11:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:44.055313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4806931-d1ef-42e7-9226-5b6f949d8c49 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:44.055601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4806931-d1ef-42e7-9226-5b6f949d8c49 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:44.055878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4806931-d1ef-42e7-9226-5b6f949d8c49 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:44.057271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 662/1320] 10.222.0.42 () {54 vars in 964 bytes} [Wed May 25 16:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:11:44.063174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdc03f-c5ec-4b06-984a-18ce7c5543e7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:44.063976 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdc03f-c5ec-4b06-984a-18ce7c5543e7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:44.064258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdc03f-c5ec-4b06-984a-18ce7c5543e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:44.085219 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:44.137733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f1bdc03f-c5ec-4b06-984a-18ce7c5543e7 None None] MFA Rules not processed for user `e42c796dbd7441289a7cbc328abeaf81`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:44.139121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f1bdc03f-c5ec-4b06-984a-18ce7c5543e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:44.216770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 659/1321] 10.222.0.42 () {58 vars in 1061 bytes} [Wed May 25 16:11:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:44.226050 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4806931-d1ef-42e7-9226-5b6f949d8c49 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:44.227743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c4806931-d1ef-42e7-9226-5b6f949d8c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:44.334543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4806931-d1ef-42e7-9226-5b6f949d8c49 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:44.411775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58d477-60f1-4afe-9de7-10f620856d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:44.412412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58d477-60f1-4afe-9de7-10f620856d68 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:44.412724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58d477-60f1-4afe-9de7-10f620856d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:44.413000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58d477-60f1-4afe-9de7-10f620856d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:44.414852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bc58d477-60f1-4afe-9de7-10f620856d68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:44.423532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:44.531282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58d477-60f1-4afe-9de7-10f620856d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:44.533412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58d477-60f1-4afe-9de7-10f620856d68 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:44.553057 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 663/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:11:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:11:47.974669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6d2a1-e6a4-4410-a669-d693e24d682e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:47.975215 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6d2a1-e6a4-4410-a669-d693e24d682e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:47.975215 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6d2a1-e6a4-4410-a669-d693e24d682e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:47.976597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 660/1323] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:11:47.981835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc73dac-bddd-42ad-aa3c-483825e522f2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:47.982246 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc73dac-bddd-42ad-aa3c-483825e522f2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:47.982392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc73dac-bddd-42ad-aa3c-483825e522f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:48.047486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8bc73dac-bddd-42ad-aa3c-483825e522f2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:48.048879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8bc73dac-bddd-42ad-aa3c-483825e522f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:48.107742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:48.148314 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 664/1324] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:48.188915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6d2a1-e6a4-4410-a669-d693e24d682e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:48.191105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-31d6d2a1-e6a4-4410-a669-d693e24d682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:48.296627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6d2a1-e6a4-4410-a669-d693e24d682e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:48.309023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:48.372060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0758fb-c554-4854-8911-94e66716f983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:48.372709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0758fb-c554-4854-8911-94e66716f983 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:48.372990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0758fb-c554-4854-8911-94e66716f983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:48.373259 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0758fb-c554-4854-8911-94e66716f983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:48.374915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5d0758fb-c554-4854-8911-94e66716f983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:48.487575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0758fb-c554-4854-8911-94e66716f983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:48.489092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0758fb-c554-4854-8911-94e66716f983 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:48.510353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 661/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:11:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:11:50.822430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0a115-c63d-4c9e-8fcd-0cfe6fd3ec6a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:50.822963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0a115-c63d-4c9e-8fcd-0cfe6fd3ec6a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:50.823044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0a115-c63d-4c9e-8fcd-0cfe6fd3ec6a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:50.824423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 665/1326] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:11:50.830883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ad3d4-1f05-4c5e-a61d-5c8c5e0841bb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:50.831386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ad3d4-1f05-4c5e-a61d-5c8c5e0841bb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:50.831480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ad3d4-1f05-4c5e-a61d-5c8c5e0841bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:50.878764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:50.898161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d48ad3d4-1f05-4c5e-a61d-5c8c5e0841bb None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:50.899343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d48ad3d4-1f05-4c5e-a61d-5c8c5e0841bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:50.996995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 662/1327] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:51.034472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0a115-c63d-4c9e-8fcd-0cfe6fd3ec6a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:51.036072 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f8f0a115-c63d-4c9e-8fcd-0cfe6fd3ec6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:51.138934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0a115-c63d-4c9e-8fcd-0cfe6fd3ec6a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:51.213611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bf381-14cc-4379-bbdd-8458017f4c6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:51.214281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bf381-14cc-4379-bbdd-8458017f4c6f service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:51.214564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bf381-14cc-4379-bbdd-8458017f4c6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:51.214843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bf381-14cc-4379-bbdd-8458017f4c6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:51.216531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-241bf381-14cc-4379-bbdd-8458017f4c6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:51.236081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:51.333111 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bf381-14cc-4379-bbdd-8458017f4c6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:51.334702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bf381-14cc-4379-bbdd-8458017f4c6f service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:51.354435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 666/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:11:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:11:53.666935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be686e-4e54-4343-9889-a83629e5fc34 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:53.667502 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be686e-4e54-4343-9889-a83629e5fc34 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:53.667502 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be686e-4e54-4343-9889-a83629e5fc34 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:53.669030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 663/1329] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:11:53.675220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1df37e-3a73-4bfd-88c6-f2a786db3ca6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:53.675635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1df37e-3a73-4bfd-88c6-f2a786db3ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:53.675843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1df37e-3a73-4bfd-88c6-f2a786db3ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:53.744416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7f1df37e-3a73-4bfd-88c6-f2a786db3ca6 None None] MFA Rules not processed for user `ab37fe904ac74e65a450dec01100babf`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:53.745650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7f1df37e-3a73-4bfd-88c6-f2a786db3ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:53.811957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:53.848871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 667/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:53.888404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be686e-4e54-4343-9889-a83629e5fc34 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:53.890093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18be686e-4e54-4343-9889-a83629e5fc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:53.999712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be686e-4e54-4343-9889-a83629e5fc34 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:54.021071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:54.076774 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcc15d-2fa1-460e-a5d4-c08fa228231c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:54.077424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcc15d-2fa1-460e-a5d4-c08fa228231c service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:54.077702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcc15d-2fa1-460e-a5d4-c08fa228231c service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:54.078004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcc15d-2fa1-460e-a5d4-c08fa228231c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:54.080506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-14dcc15d-2fa1-460e-a5d4-c08fa228231c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:54.189702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcc15d-2fa1-460e-a5d4-c08fa228231c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:54.191161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcc15d-2fa1-460e-a5d4-c08fa228231c service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:54.209154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 664/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:11:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:11:56.690476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4b5a-c7de-48bd-982b-8a83ad30e0c8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:56.691081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4b5a-c7de-48bd-982b-8a83ad30e0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:56.691081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4b5a-c7de-48bd-982b-8a83ad30e0c8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:56.692427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 668/1332] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:11:56.697624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09099f1-db05-4d08-a07a-3c2fefb118c7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:56.698117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09099f1-db05-4d08-a07a-3c2fefb118c7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:56.698222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09099f1-db05-4d08-a07a-3c2fefb118c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:56.761593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:56.764631 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d09099f1-db05-4d08-a07a-3c2fefb118c7 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:56.766940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d09099f1-db05-4d08-a07a-3c2fefb118c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:56.878795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 665/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:56.917446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e4b5a-c7de-48bd-982b-8a83ad30e0c8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:56.919034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d29e4b5a-c7de-48bd-982b-8a83ad30e0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:57.031978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e4b5a-c7de-48bd-982b-8a83ad30e0c8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:57.107391 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c82e7-774b-424c-904e-16f668fdcf45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:57.108102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c82e7-774b-424c-904e-16f668fdcf45 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:57.108329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c82e7-774b-424c-904e-16f668fdcf45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:57.108605 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c82e7-774b-424c-904e-16f668fdcf45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:57.110374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-974c82e7-774b-424c-904e-16f668fdcf45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:57.140163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:57.231932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c82e7-774b-424c-904e-16f668fdcf45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:57.233559 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c82e7-774b-424c-904e-16f668fdcf45 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:57.255058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 669/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:11:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:11:59.545777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb11cc4-9b6e-498c-9d53-7f77f3b03ab0 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:59.546300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb11cc4-9b6e-498c-9d53-7f77f3b03ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:59.546300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb11cc4-9b6e-498c-9d53-7f77f3b03ab0 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:59.548134 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 666/1335] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:11:59.553623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b96faf-7d4e-4de7-857f-28d607706b87 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:59.554037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b96faf-7d4e-4de7-857f-28d607706b87 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:59.554222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b96faf-7d4e-4de7-857f-28d607706b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:59.621082 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c1b96faf-7d4e-4de7-857f-28d607706b87 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:59.622320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c1b96faf-7d4e-4de7-857f-28d607706b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:59.700860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:59.724684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 670/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:11:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:59.764787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb11cc4-9b6e-498c-9d53-7f77f3b03ab0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:59.766339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6eb11cc4-9b6e-498c-9d53-7f77f3b03ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:59.869882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb11cc4-9b6e-498c-9d53-7f77f3b03ab0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:59.898111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:59.944860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2234e1-7335-4cba-9250-c883ea80e0cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:59.945468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2234e1-7335-4cba-9250-c883ea80e0cc service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:59.945772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2234e1-7335-4cba-9250-c883ea80e0cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:59.946050 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2234e1-7335-4cba-9250-c883ea80e0cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:59.947666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6c2234e1-7335-4cba-9250-c883ea80e0cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:00.060770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2234e1-7335-4cba-9250-c883ea80e0cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:00.062242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2234e1-7335-4cba-9250-c883ea80e0cc service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:00.084863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 667/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:11:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:02.670230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b84ea3-2cd0-460b-9d3b-2d2ac30c28a0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:02.670761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b84ea3-2cd0-460b-9d3b-2d2ac30c28a0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:02.670890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b84ea3-2cd0-460b-9d3b-2d2ac30c28a0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:02.672207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 671/1338] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:02.678017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d97b0-0bd7-4060-bdef-758350c3814d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:02.678455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d97b0-0bd7-4060-bdef-758350c3814d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:02.678586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d97b0-0bd7-4060-bdef-758350c3814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:02.743406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-421d97b0-0bd7-4060-bdef-758350c3814d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:02.744548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-421d97b0-0bd7-4060-bdef-758350c3814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:02.752784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:02.840937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 668/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:02.877038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b84ea3-2cd0-460b-9d3b-2d2ac30c28a0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:02.878681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-51b84ea3-2cd0-460b-9d3b-2d2ac30c28a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:02.982547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b84ea3-2cd0-460b-9d3b-2d2ac30c28a0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:03.055350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de526ddb-1360-4a4f-955f-dedb67bd5a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:03.055993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de526ddb-1360-4a4f-955f-dedb67bd5a88 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:03.056276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de526ddb-1360-4a4f-955f-dedb67bd5a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:03.056549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de526ddb-1360-4a4f-955f-dedb67bd5a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:03.058213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-de526ddb-1360-4a4f-955f-dedb67bd5a88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:03.094362 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:03.171653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de526ddb-1360-4a4f-955f-dedb67bd5a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:03.173204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de526ddb-1360-4a4f-955f-dedb67bd5a88 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:03.192634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 672/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:07.566007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d483f-6da0-436c-bed7-dcdd4d02024c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:07.566584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d483f-6da0-436c-bed7-dcdd4d02024c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:07.566584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d483f-6da0-436c-bed7-dcdd4d02024c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:07.567949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 669/1341] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:07.573491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e4e9c-4c25-40f1-80ae-9848810656c8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:07.574049 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e4e9c-4c25-40f1-80ae-9848810656c8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:07.574379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e4e9c-4c25-40f1-80ae-9848810656c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:07.650202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-555e4e9c-4c25-40f1-80ae-9848810656c8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:07.651428 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-555e4e9c-4c25-40f1-80ae-9848810656c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:07.749794 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:07.764243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 673/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:12:07.804484 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d483f-6da0-436c-bed7-dcdd4d02024c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:07.806724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9f8d483f-6da0-436c-bed7-dcdd4d02024c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:07.908979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d483f-6da0-436c-bed7-dcdd4d02024c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:07.912627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbe17e-b646-4497-b409-91ccf6e90cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:07.913288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbe17e-b646-4497-b409-91ccf6e90cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:07.913574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbe17e-b646-4497-b409-91ccf6e90cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:07.913893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbe17e-b646-4497-b409-91ccf6e90cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:07.924910 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbe17e-b646-4497-b409-91ccf6e90cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:07.926400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbe17e-b646-4497-b409-91ccf6e90cb4 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:07.936629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 670/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:10.857686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c4a40-4acf-4a86-880c-94e3ea0cdb91 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:10.858299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c4a40-4acf-4a86-880c-94e3ea0cdb91 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:10.858299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c4a40-4acf-4a86-880c-94e3ea0cdb91 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:10.859654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 674/1344] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:10.865800 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071e0ca-111c-4968-be51-60b634fd63b4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:10.866240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071e0ca-111c-4968-be51-60b634fd63b4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:10.866368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071e0ca-111c-4968-be51-60b634fd63b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:10.894701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a071e0ca-111c-4968-be51-60b634fd63b4 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:10.895890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a071e0ca-111c-4968-be51-60b634fd63b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:10.924175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:10.975356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 671/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:11.012942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c4a40-4acf-4a86-880c-94e3ea0cdb91 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:11.024333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c4a40-4acf-4a86-880c-94e3ea0cdb91 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:11.027998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e87513-3c3f-4cad-86c8-f9fb099e147c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:11.028714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87513-3c3f-4cad-86c8-f9fb099e147c service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:11.029070 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87513-3c3f-4cad-86c8-f9fb099e147c service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:11.029422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87513-3c3f-4cad-86c8-f9fb099e147c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:11.039895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e87513-3c3f-4cad-86c8-f9fb099e147c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:11.041343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e87513-3c3f-4cad-86c8-f9fb099e147c service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:11.051563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 675/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:15.362348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cade2-ca6a-4c98-8c61-01446b38312b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:15.362806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cade2-ca6a-4c98-8c61-01446b38312b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:15.362922 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cade2-ca6a-4c98-8c61-01446b38312b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:15.368398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 672/1347] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:15.370193 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fc47c-71b4-446d-b2ae-b241b07e34fb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:15.370640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fc47c-71b4-446d-b2ae-b241b07e34fb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:15.370780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fc47c-71b4-446d-b2ae-b241b07e34fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:15.389947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7e3fc47c-71b4-446d-b2ae-b241b07e34fb None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:15.391384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7e3fc47c-71b4-446d-b2ae-b241b07e34fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:15.401752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 676/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:15.440218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cade2-ca6a-4c98-8c61-01446b38312b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:15.452009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cade2-ca6a-4c98-8c61-01446b38312b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:15.455919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dabe2d-67db-45ed-a6c5-c903d61961d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:15.456654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dabe2d-67db-45ed-a6c5-c903d61961d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:15.457010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dabe2d-67db-45ed-a6c5-c903d61961d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:15.457354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dabe2d-67db-45ed-a6c5-c903d61961d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:15.466691 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dabe2d-67db-45ed-a6c5-c903d61961d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:15.468193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dabe2d-67db-45ed-a6c5-c903d61961d2 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:15.478769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 673/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:20.908689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398c0b4-1549-4e40-9c25-000c2bfec1d5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:20.909786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398c0b4-1549-4e40-9c25-000c2bfec1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:20.910206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398c0b4-1549-4e40-9c25-000c2bfec1d5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:20.912470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 677/1350] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:12:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:20.932238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec256f-612e-4e55-b9c5-99cc594ee6aa None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:20.933104 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec256f-612e-4e55-b9c5-99cc594ee6aa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:20.933532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec256f-612e-4e55-b9c5-99cc594ee6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:21.018780 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6aec256f-612e-4e55-b9c5-99cc594ee6aa None None] MFA Rules not processed for user `6719421144734ef49b3809df30a45961`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:21.019981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6aec256f-612e-4e55-b9c5-99cc594ee6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:21.105028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 674/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:25.384979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=13 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:25.596165 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476ecda-d38e-4ffc-a2e0-65e0e418cab5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:25.596559 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476ecda-d38e-4ffc-a2e0-65e0e418cab5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:25.596772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476ecda-d38e-4ffc-a2e0-65e0e418cab5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:25.598441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 678/1352] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:25.603602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a2305-f935-418c-91c5-22e823879a75 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:25.604580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a2305-f935-418c-91c5-22e823879a75 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:25.605195 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a2305-f935-418c-91c5-22e823879a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:25.629348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-662a2305-f935-418c-91c5-22e823879a75 None None] MFA Rules not processed for user `6719421144734ef49b3809df30a45961`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:25.631093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-662a2305-f935-418c-91c5-22e823879a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:25.642677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 675/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:12:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:27.153695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dea404-8fcf-4cc2-a841-b1279d142670 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:27.154312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dea404-8fcf-4cc2-a841-b1279d142670 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:27.154312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dea404-8fcf-4cc2-a841-b1279d142670 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:27.155910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 679/1354] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:27.162196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa728f-9a7a-4d2b-afd1-27e5545f3699 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:27.162674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa728f-9a7a-4d2b-afd1-27e5545f3699 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:27.162771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa728f-9a7a-4d2b-afd1-27e5545f3699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:27.181079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5dfa728f-9a7a-4d2b-afd1-27e5545f3699 None None] MFA Rules not processed for user `6719421144734ef49b3809df30a45961`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:27.182308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5dfa728f-9a7a-4d2b-afd1-27e5545f3699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:27.191866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 676/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:12:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:31.175042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63e65a-6977-42f7-9200-830382c522ed None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:31.175610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63e65a-6977-42f7-9200-830382c522ed None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:31.175610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63e65a-6977-42f7-9200-830382c522ed None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:31.177119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 680/1356] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:31.183043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a57a06-6f49-4686-a177-d0e93feaa715 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:31.183539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a57a06-6f49-4686-a177-d0e93feaa715 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:31.183815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a57a06-6f49-4686-a177-d0e93feaa715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:31.203094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-99a57a06-6f49-4686-a177-d0e93feaa715 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:31.204423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-99a57a06-6f49-4686-a177-d0e93feaa715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:31.213348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 677/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:31.273994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f287-6829-4b3f-9a01-fbcdeba1b480 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:31.274551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f287-6829-4b3f-9a01-fbcdeba1b480 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:31.275157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f287-6829-4b3f-9a01-fbcdeba1b480 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:31.277531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 681/1358] 10.222.0.42 () {54 vars in 961 bytes} [Wed May 25 16:12:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:31.287943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff338748-e150-4236-a54d-6f7cb3dcd67d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:31.294217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff338748-e150-4236-a54d-6f7cb3dcd67d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:31.301685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff338748-e150-4236-a54d-6f7cb3dcd67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:31.360866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:31.374040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ff338748-e150-4236-a54d-6f7cb3dcd67d None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:31.375387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ff338748-e150-4236-a54d-6f7cb3dcd67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:31.473079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 678/1359] 10.222.0.42 () {58 vars in 1058 bytes} [Wed May 25 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:31.482365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3f287-6829-4b3f-9a01-fbcdeba1b480 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:31.500302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3f287-6829-4b3f-9a01-fbcdeba1b480 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:31.506674 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4f51c-437d-4cae-890b-3c32b09dbea9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:31.507256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4f51c-437d-4cae-890b-3c32b09dbea9 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:31.507713 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4f51c-437d-4cae-890b-3c32b09dbea9 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:31.507996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4f51c-437d-4cae-890b-3c32b09dbea9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:31.520179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4f51c-437d-4cae-890b-3c32b09dbea9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:31.522161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4f51c-437d-4cae-890b-3c32b09dbea9 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:31.535844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 682/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:34.052440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d661b-6819-493e-ab06-5934ba07c12f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:34.052975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d661b-6819-493e-ab06-5934ba07c12f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:34.052975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d661b-6819-493e-ab06-5934ba07c12f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:34.054920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 679/1361] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:34.060237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed6247-e16b-4d7b-84cc-e0838a26e940 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:34.061068 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed6247-e16b-4d7b-84cc-e0838a26e940 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:34.061352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed6247-e16b-4d7b-84cc-e0838a26e940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:34.081539 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-84ed6247-e16b-4d7b-84cc-e0838a26e940 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:34.083117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-84ed6247-e16b-4d7b-84cc-e0838a26e940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:34.091844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 683/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:34.136034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d661b-6819-493e-ab06-5934ba07c12f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:34.155309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d661b-6819-493e-ab06-5934ba07c12f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:34.162151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16380851-5387-4cfb-94f6-74fbd081b087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:34.162906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16380851-5387-4cfb-94f6-74fbd081b087 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:34.163201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16380851-5387-4cfb-94f6-74fbd081b087 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:34.163478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16380851-5387-4cfb-94f6-74fbd081b087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:34.172391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16380851-5387-4cfb-94f6-74fbd081b087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:34.174031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16380851-5387-4cfb-94f6-74fbd081b087 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:34.184492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 680/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:36.790658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be64ef-d3ca-44fe-9195-f67eb0748433 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:36.791234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be64ef-d3ca-44fe-9195-f67eb0748433 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:36.791234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be64ef-d3ca-44fe-9195-f67eb0748433 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:36.792629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 684/1364] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:36.799873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f56bac-8cc3-482b-8c25-e1cc476e53c5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:36.799873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f56bac-8cc3-482b-8c25-e1cc476e53c5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:36.799873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f56bac-8cc3-482b-8c25-e1cc476e53c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:36.818989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a5f56bac-8cc3-482b-8c25-e1cc476e53c5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:36.820203 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a5f56bac-8cc3-482b-8c25-e1cc476e53c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:36.828322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 681/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:36.874148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54084922-c68d-4011-b76c-de55750ae511 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:36.874445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54084922-c68d-4011-b76c-de55750ae511 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:36.874715 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54084922-c68d-4011-b76c-de55750ae511 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:36.876814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 685/1366] 10.222.0.42 () {54 vars in 961 bytes} [Wed May 25 16:12:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:36.882536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583e585-d4ad-4000-adc5-ad4989238fd9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:36.883187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583e585-d4ad-4000-adc5-ad4989238fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:36.883187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583e585-d4ad-4000-adc5-ad4989238fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:36.902445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8583e585-d4ad-4000-adc5-ad4989238fd9 None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:36.903663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8583e585-d4ad-4000-adc5-ad4989238fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:36.914426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 682/1367] 10.222.0.42 () {58 vars in 1058 bytes} [Wed May 25 16:12:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:12:36.923277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54084922-c68d-4011-b76c-de55750ae511 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:36.942748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54084922-c68d-4011-b76c-de55750ae511 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:36.950008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f02e4-7024-4f80-81a0-6a6a75c927b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:36.951174 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f02e4-7024-4f80-81a0-6a6a75c927b9 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:36.951617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f02e4-7024-4f80-81a0-6a6a75c927b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:36.952095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f02e4-7024-4f80-81a0-6a6a75c927b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:36.966813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f02e4-7024-4f80-81a0-6a6a75c927b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:36.969442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f02e4-7024-4f80-81a0-6a6a75c927b9 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:36.982228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 686/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:39.524660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd52b5a-c39f-40dc-a076-d44dcf37fcaf None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:39.525209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd52b5a-c39f-40dc-a076-d44dcf37fcaf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:39.525209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd52b5a-c39f-40dc-a076-d44dcf37fcaf None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:39.526522 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 683/1369] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:39.532454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db83fd2-d706-48f0-a404-f44d7e0d5aa2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:39.533580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db83fd2-d706-48f0-a404-f44d7e0d5aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:39.534320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db83fd2-d706-48f0-a404-f44d7e0d5aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:39.553184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5db83fd2-d706-48f0-a404-f44d7e0d5aa2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:39.554405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5db83fd2-d706-48f0-a404-f44d7e0d5aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:39.565691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 687/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:39.611303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd52b5a-c39f-40dc-a076-d44dcf37fcaf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:39.626598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd52b5a-c39f-40dc-a076-d44dcf37fcaf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:39.632105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f63794-bbe7-4747-9ae0-5a2eaddcad8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:39.632743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f63794-bbe7-4747-9ae0-5a2eaddcad8f service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:39.633027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f63794-bbe7-4747-9ae0-5a2eaddcad8f service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:39.633303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f63794-bbe7-4747-9ae0-5a2eaddcad8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:39.643182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f63794-bbe7-4747-9ae0-5a2eaddcad8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:39.646207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f63794-bbe7-4747-9ae0-5a2eaddcad8f service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:39.660058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 684/1371] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:39.771360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33255ce2-16dc-4a0a-aa16-05ceb74e3669 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:39.771652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33255ce2-16dc-4a0a-aa16-05ceb74e3669 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:39.772168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33255ce2-16dc-4a0a-aa16-05ceb74e3669 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:39.777857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 688/1372] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:39.780121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03023306-08fc-4d9f-8193-3274731ba029 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:39.780745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03023306-08fc-4d9f-8193-3274731ba029 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:39.781106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03023306-08fc-4d9f-8193-3274731ba029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:39.803916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-03023306-08fc-4d9f-8193-3274731ba029 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:39.805336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-03023306-08fc-4d9f-8193-3274731ba029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:39.817940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 685/1373] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:12:40.129650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72d0b-a805-4f09-bb20-91eeb5063a9f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:40.129650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72d0b-a805-4f09-bb20-91eeb5063a9f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:40.129650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72d0b-a805-4f09-bb20-91eeb5063a9f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:40.131842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 689/1374] 10.222.0.42 () {54 vars in 963 bytes} [Wed May 25 16:12:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:40.138696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0087-9fd9-437d-a942-488815f8bf67 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:40.138989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0087-9fd9-437d-a942-488815f8bf67 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:40.139272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0087-9fd9-437d-a942-488815f8bf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:40.203017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6fcc0087-9fd9-437d-a942-488815f8bf67 None None] MFA Rules not processed for user `b9913b9eb502434985f84c4a11632ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:40.204391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6fcc0087-9fd9-437d-a942-488815f8bf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:40.275517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 686/1375] 10.222.0.42 () {58 vars in 1060 bytes} [Wed May 25 16:12:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:40.283583 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da72d0b-a805-4f09-bb20-91eeb5063a9f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:40.294756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da72d0b-a805-4f09-bb20-91eeb5063a9f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:40.299238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87289a4c-afbf-4850-b5cc-9103471934a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:40.299877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87289a4c-afbf-4850-b5cc-9103471934a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:40.300162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87289a4c-afbf-4850-b5cc-9103471934a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:40.300435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87289a4c-afbf-4850-b5cc-9103471934a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:40.310203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87289a4c-afbf-4850-b5cc-9103471934a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:40.311688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87289a4c-afbf-4850-b5cc-9103471934a8 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:40.322899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 690/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:12:41.950098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe6338-a414-44c5-99ad-5a2a30efa758 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:41.950684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe6338-a414-44c5-99ad-5a2a30efa758 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:41.950684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe6338-a414-44c5-99ad-5a2a30efa758 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:41.952253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 687/1377] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:41.959092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0f446-b886-41bb-9f08-e966ef8af4a0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:41.959646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0f446-b886-41bb-9f08-e966ef8af4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:41.959646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0f446-b886-41bb-9f08-e966ef8af4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:41.979315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-83f0f446-b886-41bb-9f08-e966ef8af4a0 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:41.980928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-83f0f446-b886-41bb-9f08-e966ef8af4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:41.989955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 691/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:42.036093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe6338-a414-44c5-99ad-5a2a30efa758 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:42.051324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe6338-a414-44c5-99ad-5a2a30efa758 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:42.058438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf3303-2104-4dd7-9b86-7b53ed3b07c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:42.058671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3303-2104-4dd7-9b86-7b53ed3b07c8 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:42.058912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3303-2104-4dd7-9b86-7b53ed3b07c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:42.059193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3303-2104-4dd7-9b86-7b53ed3b07c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:42.069194 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf3303-2104-4dd7-9b86-7b53ed3b07c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:42.070869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf3303-2104-4dd7-9b86-7b53ed3b07c8 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:42.075249 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:42.083209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 688/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:44.388320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752da09d-1f6e-46b1-9153-bbac95059759 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:44.388961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752da09d-1f6e-46b1-9153-bbac95059759 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:44.389163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752da09d-1f6e-46b1-9153-bbac95059759 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:44.390818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 692/1380] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:44.396745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c7733-8ee0-4e7a-b06b-479492f2b543 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:44.397231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c7733-8ee0-4e7a-b06b-479492f2b543 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:44.397317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c7733-8ee0-4e7a-b06b-479492f2b543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:44.417018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fb4c7733-8ee0-4e7a-b06b-479492f2b543 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:44.418237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fb4c7733-8ee0-4e7a-b06b-479492f2b543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:44.442414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 689/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:44.477082 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752da09d-1f6e-46b1-9153-bbac95059759 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:44.492650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752da09d-1f6e-46b1-9153-bbac95059759 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:44.498870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d82063-72ba-4442-bc8e-c484b9c8a550 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:44.499601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d82063-72ba-4442-bc8e-c484b9c8a550 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:44.499956 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d82063-72ba-4442-bc8e-c484b9c8a550 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:44.500379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d82063-72ba-4442-bc8e-c484b9c8a550 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:44.509994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d82063-72ba-4442-bc8e-c484b9c8a550 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:44.511736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d82063-72ba-4442-bc8e-c484b9c8a550 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:44.522721 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 693/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:46.808410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a9368-b8cc-4074-8fcc-df0186111c7f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:46.808939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a9368-b8cc-4074-8fcc-df0186111c7f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:46.809426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a9368-b8cc-4074-8fcc-df0186111c7f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:46.811260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 690/1383] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:46.817572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd7f1e-ce41-4427-b85b-33eeb1a249d3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:46.818089 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd7f1e-ce41-4427-b85b-33eeb1a249d3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:46.818171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd7f1e-ce41-4427-b85b-33eeb1a249d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:46.837392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-69bd7f1e-ce41-4427-b85b-33eeb1a249d3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:46.838878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-69bd7f1e-ce41-4427-b85b-33eeb1a249d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:46.848360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 694/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:46.890021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a9368-b8cc-4074-8fcc-df0186111c7f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:46.910156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a9368-b8cc-4074-8fcc-df0186111c7f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:46.916217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c844a7-bc99-4f08-a19c-09051636cede service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:46.916991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c844a7-bc99-4f08-a19c-09051636cede service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:46.917280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c844a7-bc99-4f08-a19c-09051636cede service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:46.917556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c844a7-bc99-4f08-a19c-09051636cede service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:46.928198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c844a7-bc99-4f08-a19c-09051636cede service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:46.929975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c844a7-bc99-4f08-a19c-09051636cede service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:46.946293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 691/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:49.218443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4b9fb-9b44-41da-a618-772a40f7efa1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:49.219499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4b9fb-9b44-41da-a618-772a40f7efa1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:49.219499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4b9fb-9b44-41da-a618-772a40f7efa1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:49.220655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 695/1386] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:49.227346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafdca8-1550-473f-8357-560233c70ce5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:49.228157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafdca8-1550-473f-8357-560233c70ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:49.228157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafdca8-1550-473f-8357-560233c70ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:49.246638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9eafdca8-1550-473f-8357-560233c70ce5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:49.247908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9eafdca8-1550-473f-8357-560233c70ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:49.257420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 692/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:49.310825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4b9fb-9b44-41da-a618-772a40f7efa1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:49.326562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4b9fb-9b44-41da-a618-772a40f7efa1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:49.333367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f11b5-b96c-476a-81ef-d80b9ea129cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:49.334078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f11b5-b96c-476a-81ef-d80b9ea129cb service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:49.334362 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f11b5-b96c-476a-81ef-d80b9ea129cb service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:49.334654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f11b5-b96c-476a-81ef-d80b9ea129cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:49.343558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f11b5-b96c-476a-81ef-d80b9ea129cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:49.345273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f11b5-b96c-476a-81ef-d80b9ea129cb service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:49.355082 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 696/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:51.861405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e2362-e2a4-4cf1-9b71-7327931974a5 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:51.861954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e2362-e2a4-4cf1-9b71-7327931974a5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:51.862043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e2362-e2a4-4cf1-9b71-7327931974a5 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:51.863379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 693/1389] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:51.869011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f56832-4fc0-4f1c-8fe1-7c5044fa6541 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:51.869459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f56832-4fc0-4f1c-8fe1-7c5044fa6541 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:51.869602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f56832-4fc0-4f1c-8fe1-7c5044fa6541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:51.889673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-47f56832-4fc0-4f1c-8fe1-7c5044fa6541 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:51.891025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-47f56832-4fc0-4f1c-8fe1-7c5044fa6541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:51.899241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 697/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:51.948263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e2362-e2a4-4cf1-9b71-7327931974a5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:51.963767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e2362-e2a4-4cf1-9b71-7327931974a5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:51.969838 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fadfaa-cbf4-49ac-865d-f761d1eeefea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:51.970561 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fadfaa-cbf4-49ac-865d-f761d1eeefea service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:51.970858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fadfaa-cbf4-49ac-865d-f761d1eeefea service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:51.971135 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fadfaa-cbf4-49ac-865d-f761d1eeefea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:51.980196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fadfaa-cbf4-49ac-865d-f761d1eeefea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:51.981894 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fadfaa-cbf4-49ac-865d-f761d1eeefea service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:51.993117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 694/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:54.743168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfcac4-322c-4e2d-9a02-d6b1deb64d5a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:54.743727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfcac4-322c-4e2d-9a02-d6b1deb64d5a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:54.743727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfcac4-322c-4e2d-9a02-d6b1deb64d5a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:54.745139 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 698/1392] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:12:54.751555 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff0be0-5b6d-4dc5-8b0a-a9b78ab64dce None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:54.752041 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff0be0-5b6d-4dc5-8b0a-a9b78ab64dce None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:54.752193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff0be0-5b6d-4dc5-8b0a-a9b78ab64dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:54.769991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1eff0be0-5b6d-4dc5-8b0a-a9b78ab64dce None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:54.771176 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1eff0be0-5b6d-4dc5-8b0a-a9b78ab64dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:54.778879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 695/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:54.825765 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfcac4-322c-4e2d-9a02-d6b1deb64d5a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:54.840308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfcac4-322c-4e2d-9a02-d6b1deb64d5a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:54.846006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ac216-4c59-49d8-b59e-4220ed437f29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:54.846646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ac216-4c59-49d8-b59e-4220ed437f29 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:54.846932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ac216-4c59-49d8-b59e-4220ed437f29 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:54.847209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ac216-4c59-49d8-b59e-4220ed437f29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:54.855580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ac216-4c59-49d8-b59e-4220ed437f29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:54.857176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ac216-4c59-49d8-b59e-4220ed437f29 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:54.867554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 699/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:56.381098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1d4f-be8f-44ca-b046-8ddcff28cd2b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:56.381682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1d4f-be8f-44ca-b046-8ddcff28cd2b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:56.381682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1d4f-be8f-44ca-b046-8ddcff28cd2b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:56.383133 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 696/1395] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:56.388760 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3253c42-f784-407c-b3d2-72d51fa5610e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:56.389199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3253c42-f784-407c-b3d2-72d51fa5610e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:56.389389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3253c42-f784-407c-b3d2-72d51fa5610e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:56.395871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=52 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:56.408707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d3253c42-f784-407c-b3d2-72d51fa5610e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:56.410192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d3253c42-f784-407c-b3d2-72d51fa5610e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:56.418143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 700/1396] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:56.476157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f2def-554e-4755-9e85-fb1a2b4f011e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:56.476557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f2def-554e-4755-9e85-fb1a2b4f011e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:56.476909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f2def-554e-4755-9e85-fb1a2b4f011e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:56.478776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 697/1397] 10.222.0.42 () {54 vars in 962 bytes} [Wed May 25 16:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:12:56.483991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2b7c-ec19-4dd8-a52d-28fd4d1b5f10 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:56.484279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2b7c-ec19-4dd8-a52d-28fd4d1b5f10 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:56.485181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2b7c-ec19-4dd8-a52d-28fd4d1b5f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:56.536182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-650c2b7c-ec19-4dd8-a52d-28fd4d1b5f10 None None] MFA Rules not processed for user `8386ca93fe48460ba5b6f0c10dc51122`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:56.537391 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-650c2b7c-ec19-4dd8-a52d-28fd4d1b5f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:56.606383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 701/1398] 10.222.0.42 () {58 vars in 1059 bytes} [Wed May 25 16:12:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:56.614872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f2def-554e-4755-9e85-fb1a2b4f011e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:56.625918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f2def-554e-4755-9e85-fb1a2b4f011e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:56.629675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e632e9a-c0be-465f-b16d-ff6c75f107d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8386ca93fe48460ba5b6f0c10dc51122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:56.630343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e632e9a-c0be-465f-b16d-ff6c75f107d8 service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:56.630623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e632e9a-c0be-465f-b16d-ff6c75f107d8 service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:56.630892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e632e9a-c0be-465f-b16d-ff6c75f107d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:56.640147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e632e9a-c0be-465f-b16d-ff6c75f107d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:56.641962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e632e9a-c0be-465f-b16d-ff6c75f107d8 service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:56.651372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 698/1399] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:12:57.168117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112db07-c318-4565-84ef-c0be1c6347ac None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:57.168408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112db07-c318-4565-84ef-c0be1c6347ac None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:57.168689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112db07-c318-4565-84ef-c0be1c6347ac None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:57.170175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 702/1400] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:57.176473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab030a83-362b-426f-a0d4-dc2a876cdfe6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:57.176710 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab030a83-362b-426f-a0d4-dc2a876cdfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:57.176989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab030a83-362b-426f-a0d4-dc2a876cdfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:57.201089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ab030a83-362b-426f-a0d4-dc2a876cdfe6 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:57.202207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ab030a83-362b-426f-a0d4-dc2a876cdfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:57.212206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 699/1401] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:57.223730 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea236e-3f83-40b8-ac81-7139744e2b82 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:57.224101 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea236e-3f83-40b8-ac81-7139744e2b82 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:57.224447 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea236e-3f83-40b8-ac81-7139744e2b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:57.259884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab030a83-362b-426f-a0d4-dc2a876cdfe6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:57.273672 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab030a83-362b-426f-a0d4-dc2a876cdfe6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:57.274773 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-59ea236e-3f83-40b8-ac81-7139744e2b82 None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:57.276052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-59ea236e-3f83-40b8-ac81-7139744e2b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:57.279079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7371f2b-0d98-4bbb-b0bb-60442fc0a433 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:57.279729 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7371f2b-0d98-4bbb-b0bb-60442fc0a433 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:57.280014 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7371f2b-0d98-4bbb-b0bb-60442fc0a433 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:57.280430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7371f2b-0d98-4bbb-b0bb-60442fc0a433 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:57.290189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7371f2b-0d98-4bbb-b0bb-60442fc0a433 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:57.291860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7371f2b-0d98-4bbb-b0bb-60442fc0a433 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:57.301992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 700/1402] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:57.348226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 703/1403] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:12:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:57.397466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4e732-533f-4e11-b817-4ac28a3d8368 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:57.398112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4e732-533f-4e11-b817-4ac28a3d8368 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:57.398112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4e732-533f-4e11-b817-4ac28a3d8368 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:57.399756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 701/1404] 10.222.0.42 () {54 vars in 942 bytes} [Wed May 25 16:12:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:12:57.407326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2e38b-5c02-4d0b-9d48-a39af884d12d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:57.407963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2e38b-5c02-4d0b-9d48-a39af884d12d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:57.407963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2e38b-5c02-4d0b-9d48-a39af884d12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:57.469238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e9d2e38b-5c02-4d0b-9d48-a39af884d12d None None] MFA Rules not processed for user `c6eb13611d0d4b6abe035c9bfbfde79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:57.470517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e9d2e38b-5c02-4d0b-9d48-a39af884d12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:57.484513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:57.551743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 704/1405] 10.222.0.42 () {58 vars in 1039 bytes} [Wed May 25 16:12:57 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:57.561337 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4e732-533f-4e11-b817-4ac28a3d8368 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:57.575284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4e732-533f-4e11-b817-4ac28a3d8368 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:57.581259 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551c006-87c0-4b3b-b29c-713c6801f621 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:57.581951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551c006-87c0-4b3b-b29c-713c6801f621 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:57.582230 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551c006-87c0-4b3b-b29c-713c6801f621 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:57.582500 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551c006-87c0-4b3b-b29c-713c6801f621 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:57.591300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551c006-87c0-4b3b-b29c-713c6801f621 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:57.592876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551c006-87c0-4b3b-b29c-713c6801f621 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:57.601496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 702/1406] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:12:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:12:59.674350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa479c03-54db-450d-8073-054675e408ad None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:59.674908 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa479c03-54db-450d-8073-054675e408ad None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:59.674908 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa479c03-54db-450d-8073-054675e408ad None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:59.677316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 705/1407] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:12:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:12:59.681766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20f8e-86d8-4248-8fd5-0b59cc26bdea None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:59.682247 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20f8e-86d8-4248-8fd5-0b59cc26bdea None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:59.682360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20f8e-86d8-4248-8fd5-0b59cc26bdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:59.703980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e7e20f8e-86d8-4248-8fd5-0b59cc26bdea None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:59.705635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e7e20f8e-86d8-4248-8fd5-0b59cc26bdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:59.722263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 703/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:59.761137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa479c03-54db-450d-8073-054675e408ad None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:59.776799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa479c03-54db-450d-8073-054675e408ad None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:59.783274 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237b88a-0d33-47f6-aeaa-5eb8abd9195d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:59.783934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b88a-0d33-47f6-aeaa-5eb8abd9195d service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:59.784667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b88a-0d33-47f6-aeaa-5eb8abd9195d service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:59.784970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b88a-0d33-47f6-aeaa-5eb8abd9195d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:59.794637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237b88a-0d33-47f6-aeaa-5eb8abd9195d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:59.796372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237b88a-0d33-47f6-aeaa-5eb8abd9195d service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:59.807485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 706/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:12:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:09.698235 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=46 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:09.807325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=30 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:21.185849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb07785-8e81-41ef-84c9-efe77cd79020 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.186430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb07785-8e81-41ef-84c9-efe77cd79020 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.186430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb07785-8e81-41ef-84c9-efe77cd79020 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.188755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 704/1410] 10.222.0.42 () {54 vars in 918 bytes} [Wed May 25 16:13:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:21.194153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c663a-925c-4f44-8829-92088270a6b8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.194652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c663a-925c-4f44-8829-92088270a6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.194759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c663a-925c-4f44-8829-92088270a6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.213091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f70c663a-925c-4f44-8829-92088270a6b8 None None] MFA Rules not processed for user `6719421144734ef49b3809df30a45961`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:21.214357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f70c663a-925c-4f44-8829-92088270a6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:21.231037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 707/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Wed May 25 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:13:21.246353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e8d8-4c62-44a7-8e43-99864bc78f0f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.246723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e8d8-4c62-44a7-8e43-99864bc78f0f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.246927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e8d8-4c62-44a7-8e43-99864bc78f0f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.248306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 705/1412] 10.222.0.42 () {54 vars in 918 bytes} [Wed May 25 16:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:21.254763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c53ee-babd-4b9b-8712-c91188935252 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.255097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c53ee-babd-4b9b-8712-c91188935252 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.255838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c53ee-babd-4b9b-8712-c91188935252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.276185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5f6c53ee-babd-4b9b-8712-c91188935252 None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:21.277395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5f6c53ee-babd-4b9b-8712-c91188935252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:21.286987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 708/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Wed May 25 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:24.002648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1efc6-d08c-4928-941a-31fd09b12d14 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:24.003238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1efc6-d08c-4928-941a-31fd09b12d14 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:24.003238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1efc6-d08c-4928-941a-31fd09b12d14 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:24.004645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:24.010081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df267-e3c7-4240-b2f3-cbb7cc86ac57 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:24.010524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df267-e3c7-4240-b2f3-cbb7cc86ac57 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:24.010631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df267-e3c7-4240-b2f3-cbb7cc86ac57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:24.028141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a56df267-e3c7-4240-b2f3-cbb7cc86ac57 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:24.029330 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a56df267-e3c7-4240-b2f3-cbb7cc86ac57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:24.038002 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:24.082024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1efc6-d08c-4928-941a-31fd09b12d14 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:24.097409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1efc6-d08c-4928-941a-31fd09b12d14 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:24.102891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eea358-1bdf-4946-980a-0b9caf4f7f07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:24.103517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea358-1bdf-4946-980a-0b9caf4f7f07 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:24.103796 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea358-1bdf-4946-980a-0b9caf4f7f07 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:24.104064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea358-1bdf-4946-980a-0b9caf4f7f07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:24.113050 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eea358-1bdf-4946-980a-0b9caf4f7f07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:24.114654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eea358-1bdf-4946-980a-0b9caf4f7f07 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:24.123690 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 707/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:25.100970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8189c87-de7c-434b-a201-01f53219b32a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:25.101889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8189c87-de7c-434b-a201-01f53219b32a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:25.102185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8189c87-de7c-434b-a201-01f53219b32a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:25.103623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:13:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:25.108476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c599d-ea3c-4034-aca0-455f183e267f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:25.108862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c599d-ea3c-4034-aca0-455f183e267f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:25.109839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c599d-ea3c-4034-aca0-455f183e267f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:25.127681 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-117c599d-ea3c-4034-aca0-455f183e267f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:25.128858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-117c599d-ea3c-4034-aca0-455f183e267f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:25.136518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:25.323900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8189c87-de7c-434b-a201-01f53219b32a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:25.338910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8189c87-de7c-434b-a201-01f53219b32a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:25.344529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa1011-95c4-4a74-9645-8be0cb943583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:25.345166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa1011-95c4-4a74-9645-8be0cb943583 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:25.345450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa1011-95c4-4a74-9645-8be0cb943583 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:25.345808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa1011-95c4-4a74-9645-8be0cb943583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:25.354028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa1011-95c4-4a74-9645-8be0cb943583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:25.355554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa1011-95c4-4a74-9645-8be0cb943583 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:25.364927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 711/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:26.321889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38b498-408b-47e4-bff9-241ba6b81a40 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:26.322467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38b498-408b-47e4-bff9-241ba6b81a40 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:26.322467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38b498-408b-47e4-bff9-241ba6b81a40 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:26.323857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:26.329092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95126c1-352d-4d3d-af32-ecce8a85e7a6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:26.329523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95126c1-352d-4d3d-af32-ecce8a85e7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:26.329654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95126c1-352d-4d3d-af32-ecce8a85e7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:26.347235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e95126c1-352d-4d3d-af32-ecce8a85e7a6 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:26.348435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e95126c1-352d-4d3d-af32-ecce8a85e7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:26.356132 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:26.509847 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38b498-408b-47e4-bff9-241ba6b81a40 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:26.524353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38b498-408b-47e4-bff9-241ba6b81a40 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:26.530011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48ee4b-0c23-4882-ba9e-1cd0de622e26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:26.530643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48ee4b-0c23-4882-ba9e-1cd0de622e26 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:26.530985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48ee4b-0c23-4882-ba9e-1cd0de622e26 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:26.531264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48ee4b-0c23-4882-ba9e-1cd0de622e26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:26.539707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48ee4b-0c23-4882-ba9e-1cd0de622e26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:26.541290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48ee4b-0c23-4882-ba9e-1cd0de622e26 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:26.551174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 710/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:27.536649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff436bd4-63e7-4cac-ae0f-e158d6063914 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:27.537187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff436bd4-63e7-4cac-ae0f-e158d6063914 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:27.537187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff436bd4-63e7-4cac-ae0f-e158d6063914 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:27.538539 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:27.543469 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2073818-c252-4826-a7f7-b4862e3ff7ae None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:27.543893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2073818-c252-4826-a7f7-b4862e3ff7ae None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:27.544074 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2073818-c252-4826-a7f7-b4862e3ff7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:27.562707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b2073818-c252-4826-a7f7-b4862e3ff7ae None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:27.564032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b2073818-c252-4826-a7f7-b4862e3ff7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:27.572306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:27.725602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff436bd4-63e7-4cac-ae0f-e158d6063914 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:27.740069 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff436bd4-63e7-4cac-ae0f-e158d6063914 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:27.745767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7903b6-95ec-4fde-a6a7-7cd0de2ae324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:27.746404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7903b6-95ec-4fde-a6a7-7cd0de2ae324 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:27.746690 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7903b6-95ec-4fde-a6a7-7cd0de2ae324 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:27.746965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7903b6-95ec-4fde-a6a7-7cd0de2ae324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:27.755160 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7903b6-95ec-4fde-a6a7-7cd0de2ae324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:27.756752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7903b6-95ec-4fde-a6a7-7cd0de2ae324 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:27.766575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 714/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:28.688005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5e77-e321-4626-b705-da7ee5d370ef None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:28.688528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5e77-e321-4626-b705-da7ee5d370ef None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:28.688528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5e77-e321-4626-b705-da7ee5d370ef None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:28.689918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:28.695040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ee99a-99b2-4f61-91be-e16b657516f0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:28.695487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ee99a-99b2-4f61-91be-e16b657516f0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:28.695619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ee99a-99b2-4f61-91be-e16b657516f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:28.713499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fa2ee99a-99b2-4f61-91be-e16b657516f0 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:28.714720 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fa2ee99a-99b2-4f61-91be-e16b657516f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:28.722620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:28.877187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a5e77-e321-4626-b705-da7ee5d370ef None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:28.891452 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a5e77-e321-4626-b705-da7ee5d370ef None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:28.896776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfd7a5-9b27-4dfa-b3e8-e965ca949f30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:28.897406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfd7a5-9b27-4dfa-b3e8-e965ca949f30 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:28.897689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfd7a5-9b27-4dfa-b3e8-e965ca949f30 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:28.898012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfd7a5-9b27-4dfa-b3e8-e965ca949f30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:28.906746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfd7a5-9b27-4dfa-b3e8-e965ca949f30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:28.908489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfd7a5-9b27-4dfa-b3e8-e965ca949f30 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:28.918322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 713/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:29.786971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aefa17-addf-42d8-b8ff-f083515910cb None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:29.787521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aefa17-addf-42d8-b8ff-f083515910cb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:29.787521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aefa17-addf-42d8-b8ff-f083515910cb None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:29.788919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:29.793913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0b32-374c-4c3e-b7c7-85c94a07e3fd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:29.794370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0b32-374c-4c3e-b7c7-85c94a07e3fd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:29.794480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0b32-374c-4c3e-b7c7-85c94a07e3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:29.812688 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6f3f0b32-374c-4c3e-b7c7-85c94a07e3fd None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:29.813993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6f3f0b32-374c-4c3e-b7c7-85c94a07e3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:29.821661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:29.973061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aefa17-addf-42d8-b8ff-f083515910cb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:29.989306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aefa17-addf-42d8-b8ff-f083515910cb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:29.995322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd8d2e-3a06-4e46-b6d5-599e5c6d91c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:29.996242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd8d2e-3a06-4e46-b6d5-599e5c6d91c4 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:29.996740 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd8d2e-3a06-4e46-b6d5-599e5c6d91c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:29.997248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd8d2e-3a06-4e46-b6d5-599e5c6d91c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:30.007262 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd8d2e-3a06-4e46-b6d5-599e5c6d91c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:30.008891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd8d2e-3a06-4e46-b6d5-599e5c6d91c4 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:30.019116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 717/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:30.901885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53351b2-ac88-404c-b1af-3fe0b680dffd None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:30.902455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53351b2-ac88-404c-b1af-3fe0b680dffd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:30.902455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53351b2-ac88-404c-b1af-3fe0b680dffd None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:30.903851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:30.909538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a740c1-9c2e-4b31-b7f5-e9ec064de336 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:30.910213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a740c1-9c2e-4b31-b7f5-e9ec064de336 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:30.910389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a740c1-9c2e-4b31-b7f5-e9ec064de336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:30.928035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-68a740c1-9c2e-4b31-b7f5-e9ec064de336 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:30.929234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-68a740c1-9c2e-4b31-b7f5-e9ec064de336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:30.937132 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:31.092306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53351b2-ac88-404c-b1af-3fe0b680dffd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:31.107157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53351b2-ac88-404c-b1af-3fe0b680dffd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:31.112643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d5535-257f-4a08-953d-bf795dd8a7e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:31.113283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d5535-257f-4a08-953d-bf795dd8a7e7 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.113569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d5535-257f-4a08-953d-bf795dd8a7e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.113883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d5535-257f-4a08-953d-bf795dd8a7e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.122103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d5535-257f-4a08-953d-bf795dd8a7e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:31.123685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d5535-257f-4a08-953d-bf795dd8a7e7 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:31.133569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 716/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:31.994825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f4f51-e47f-4602-80a5-c29b9439140f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.995368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f4f51-e47f-4602-80a5-c29b9439140f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.995464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f4f51-e47f-4602-80a5-c29b9439140f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.996849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:32.002549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3f30d-89a7-432e-8f57-d9857c81fe84 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.002978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3f30d-89a7-432e-8f57-d9857c81fe84 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.003107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3f30d-89a7-432e-8f57-d9857c81fe84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.021546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6fa3f30d-89a7-432e-8f57-d9857c81fe84 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:32.022769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6fa3f30d-89a7-432e-8f57-d9857c81fe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:32.030473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:32.183292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f4f51-e47f-4602-80a5-c29b9439140f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.199560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f4f51-e47f-4602-80a5-c29b9439140f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.205974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb781f-b8e0-4369-ad95-94d360d184df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.206617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb781f-b8e0-4369-ad95-94d360d184df service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.206897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb781f-b8e0-4369-ad95-94d360d184df service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.207172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb781f-b8e0-4369-ad95-94d360d184df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.215889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb781f-b8e0-4369-ad95-94d360d184df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.217638 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb781f-b8e0-4369-ad95-94d360d184df service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.227512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 720/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:33.085615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925c597-b3f9-4bdd-9e97-c153f9b51a16 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:33.086201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925c597-b3f9-4bdd-9e97-c153f9b51a16 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:33.086201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925c597-b3f9-4bdd-9e97-c153f9b51a16 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:33.087595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:33.092609 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304e80c-bd15-4615-86bd-483aad52d404 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:33.093041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304e80c-bd15-4615-86bd-483aad52d404 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:33.093192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304e80c-bd15-4615-86bd-483aad52d404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:33.111854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2304e80c-bd15-4615-86bd-483aad52d404 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:33.113067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2304e80c-bd15-4615-86bd-483aad52d404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:33.120784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:33.269876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925c597-b3f9-4bdd-9e97-c153f9b51a16 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:33.284803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925c597-b3f9-4bdd-9e97-c153f9b51a16 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:33.290772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f5215-cdb6-4040-8b36-cdd9c12863fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:33.291223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5215-cdb6-4040-8b36-cdd9c12863fc service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:33.291508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5215-cdb6-4040-8b36-cdd9c12863fc service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:33.291781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5215-cdb6-4040-8b36-cdd9c12863fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:33.301282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f5215-cdb6-4040-8b36-cdd9c12863fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:33.302898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f5215-cdb6-4040-8b36-cdd9c12863fc service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:33.312883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 719/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:35.453474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b918dd3-f5b7-4f39-984c-abdf376294aa None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:35.454094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b918dd3-f5b7-4f39-984c-abdf376294aa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:35.454094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b918dd3-f5b7-4f39-984c-abdf376294aa None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:35.455452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:13:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:35.460876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3f0c0-3392-4b3a-9ddc-e441e77015e8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:35.461302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3f0c0-3392-4b3a-9ddc-e441e77015e8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:35.461430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3f0c0-3392-4b3a-9ddc-e441e77015e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:35.479592 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b4f3f0c0-3392-4b3a-9ddc-e441e77015e8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:35.480722 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b4f3f0c0-3392-4b3a-9ddc-e441e77015e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:35.488672 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:13:35.527158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b918dd3-f5b7-4f39-984c-abdf376294aa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:35.538312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b918dd3-f5b7-4f39-984c-abdf376294aa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:35.541759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645c629-0f49-4fa0-b93a-90aad46da25c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:35.542402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645c629-0f49-4fa0-b93a-90aad46da25c service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:35.542685 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645c629-0f49-4fa0-b93a-90aad46da25c service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:35.542964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645c629-0f49-4fa0-b93a-90aad46da25c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:35.552443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645c629-0f49-4fa0-b93a-90aad46da25c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:35.554003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645c629-0f49-4fa0-b93a-90aad46da25c service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:35.564052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 723/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:37.953119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7db39-f3fe-4aec-96de-c88035702fe0 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:37.953659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7db39-f3fe-4aec-96de-c88035702fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:37.953836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7db39-f3fe-4aec-96de-c88035702fe0 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:37.955237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:37.960826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc839646-7f9f-4717-8a96-33d014c90f06 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:37.961287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc839646-7f9f-4717-8a96-33d014c90f06 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:37.961415 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc839646-7f9f-4717-8a96-33d014c90f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:37.979712 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dc839646-7f9f-4717-8a96-33d014c90f06 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:37.980916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dc839646-7f9f-4717-8a96-33d014c90f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:37.989476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:13:38.039248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7db39-f3fe-4aec-96de-c88035702fe0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:38.040960 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-94a7db39-f3fe-4aec-96de-c88035702fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:38.054629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7db39-f3fe-4aec-96de-c88035702fe0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:38.058091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe26ed0-85f6-419b-b4da-69429235ba08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:38.058724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe26ed0-85f6-419b-b4da-69429235ba08 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:38.059004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe26ed0-85f6-419b-b4da-69429235ba08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:38.059277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe26ed0-85f6-419b-b4da-69429235ba08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:38.068752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe26ed0-85f6-419b-b4da-69429235ba08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:38.070198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe26ed0-85f6-419b-b4da-69429235ba08 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:38.079527 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 722/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:40.856337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc903e-743c-4d95-9425-e983d3a039a1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:40.856972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc903e-743c-4d95-9425-e983d3a039a1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:40.856972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc903e-743c-4d95-9425-e983d3a039a1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:40.858326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:40.863802 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2f4bc-837b-4595-8a42-aea0e022fa26 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:40.864225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2f4bc-837b-4595-8a42-aea0e022fa26 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:40.864431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2f4bc-837b-4595-8a42-aea0e022fa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:40.882525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-66a2f4bc-837b-4595-8a42-aea0e022fa26 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:40.883701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-66a2f4bc-837b-4595-8a42-aea0e022fa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:40.891714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:40.931129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc903e-743c-4d95-9425-e983d3a039a1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:40.941828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc903e-743c-4d95-9425-e983d3a039a1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:40.945300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b36324-c6fb-4384-8480-d621b9751e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:40.945964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b36324-c6fb-4384-8480-d621b9751e89 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:40.946249 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b36324-c6fb-4384-8480-d621b9751e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:40.946523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b36324-c6fb-4384-8480-d621b9751e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:40.955601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b36324-c6fb-4384-8480-d621b9751e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:40.957029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b36324-c6fb-4384-8480-d621b9751e89 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:40.966298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 726/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:43.354172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5906a-eed5-49a9-b44d-cd10f238ecad None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:43.354719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5906a-eed5-49a9-b44d-cd10f238ecad None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:43.354719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5906a-eed5-49a9-b44d-cd10f238ecad None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:43.356119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:43.362229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b2620-38e6-46ac-bc1f-fab4b916d4a3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:43.362689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b2620-38e6-46ac-bc1f-fab4b916d4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:43.362822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b2620-38e6-46ac-bc1f-fab4b916d4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:43.381256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5c4b2620-38e6-46ac-bc1f-fab4b916d4a3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:43.382501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5c4b2620-38e6-46ac-bc1f-fab4b916d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:43.391111 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:43.430618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5906a-eed5-49a9-b44d-cd10f238ecad None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:43.442331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5906a-eed5-49a9-b44d-cd10f238ecad None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:43.445901 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fb523-91cd-497d-8a3e-3a83dec992cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:43.446544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fb523-91cd-497d-8a3e-3a83dec992cc service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:43.446829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fb523-91cd-497d-8a3e-3a83dec992cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:43.447106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fb523-91cd-497d-8a3e-3a83dec992cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:43.456797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fb523-91cd-497d-8a3e-3a83dec992cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:43.458272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fb523-91cd-497d-8a3e-3a83dec992cc service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:43.468105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 725/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:45.829973 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d521ba6-d888-450a-aacb-848da2f18851 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:45.830538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d521ba6-d888-450a-aacb-848da2f18851 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:45.830538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d521ba6-d888-450a-aacb-848da2f18851 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:45.831926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:45.837251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc288db-e7d6-4296-922d-241f0cdfcf01 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:45.837659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc288db-e7d6-4296-922d-241f0cdfcf01 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:45.837857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc288db-e7d6-4296-922d-241f0cdfcf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:45.855993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9bc288db-e7d6-4296-922d-241f0cdfcf01 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:45.857197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9bc288db-e7d6-4296-922d-241f0cdfcf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:45.865325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:45.902859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d521ba6-d888-450a-aacb-848da2f18851 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:45.914328 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d521ba6-d888-450a-aacb-848da2f18851 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:45.917833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcd52c-1701-403e-82bc-d8627924c124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:45.918472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd52c-1701-403e-82bc-d8627924c124 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:45.918758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd52c-1701-403e-82bc-d8627924c124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:45.919037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd52c-1701-403e-82bc-d8627924c124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:45.928048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcd52c-1701-403e-82bc-d8627924c124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:45.929482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcd52c-1701-403e-82bc-d8627924c124 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:45.939010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 729/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:13:48.315136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e682c-f778-4001-b4eb-91ca7b972979 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:48.315706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e682c-f778-4001-b4eb-91ca7b972979 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:48.315706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e682c-f778-4001-b4eb-91ca7b972979 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:48.317079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:48.322573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844685-0555-4fdf-83a8-248bf45e6ca0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:48.322947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844685-0555-4fdf-83a8-248bf45e6ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:48.323138 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844685-0555-4fdf-83a8-248bf45e6ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:48.340986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-47844685-0555-4fdf-83a8-248bf45e6ca0 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:48.342224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-47844685-0555-4fdf-83a8-248bf45e6ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:48.350503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:48.389535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e682c-f778-4001-b4eb-91ca7b972979 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:48.400614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e682c-f778-4001-b4eb-91ca7b972979 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:48.404250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8687b-11ba-4b61-b522-29eb40b98b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:48.404886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8687b-11ba-4b61-b522-29eb40b98b92 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:48.405170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8687b-11ba-4b61-b522-29eb40b98b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:48.405992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8687b-11ba-4b61-b522-29eb40b98b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:48.415005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8687b-11ba-4b61-b522-29eb40b98b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:48.416468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8687b-11ba-4b61-b522-29eb40b98b92 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:48.426680 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 728/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:50.712396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4747e-06ef-4415-819c-e5b7bb6c10d1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:50.712951 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4747e-06ef-4415-819c-e5b7bb6c10d1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:50.712951 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4747e-06ef-4415-819c-e5b7bb6c10d1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:50.714256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:50.720208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b1a77-5fa7-4b3a-87b6-520cbb02f3d5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:50.720620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b1a77-5fa7-4b3a-87b6-520cbb02f3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:50.720769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b1a77-5fa7-4b3a-87b6-520cbb02f3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:50.727597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=70 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:50.742004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bd7b1a77-5fa7-4b3a-87b6-520cbb02f3d5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:50.743198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bd7b1a77-5fa7-4b3a-87b6-520cbb02f3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:50.751102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:50.788301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4747e-06ef-4415-819c-e5b7bb6c10d1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:50.793872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=73 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:50.800934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4747e-06ef-4415-819c-e5b7bb6c10d1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:50.804727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ec0bd-52e5-4004-a3f0-9a1618d7bc77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:50.805358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec0bd-52e5-4004-a3f0-9a1618d7bc77 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:50.805797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec0bd-52e5-4004-a3f0-9a1618d7bc77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:50.806094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec0bd-52e5-4004-a3f0-9a1618d7bc77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:50.815088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ec0bd-52e5-4004-a3f0-9a1618d7bc77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:50.816554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ec0bd-52e5-4004-a3f0-9a1618d7bc77 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:50.825915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 732/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:53.096187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c8bdb-1260-4e51-be0d-b0b9163593f8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:53.096738 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c8bdb-1260-4e51-be0d-b0b9163593f8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:53.096864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c8bdb-1260-4e51-be0d-b0b9163593f8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:53.098194 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:53.103658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b0ebd-b8f5-47f7-8304-e8f24248f1d5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:53.104104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b0ebd-b8f5-47f7-8304-e8f24248f1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:53.104204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b0ebd-b8f5-47f7-8304-e8f24248f1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:53.121563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7b6b0ebd-b8f5-47f7-8304-e8f24248f1d5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:53.122770 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b6b0ebd-b8f5-47f7-8304-e8f24248f1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:53.131228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:53.168911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c8bdb-1260-4e51-be0d-b0b9163593f8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:53.179589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c8bdb-1260-4e51-be0d-b0b9163593f8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:53.183156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cbf3fe-acad-404a-a4c9-47296f62a765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:53.183786 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbf3fe-acad-404a-a4c9-47296f62a765 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:53.184075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbf3fe-acad-404a-a4c9-47296f62a765 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:53.184352 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbf3fe-acad-404a-a4c9-47296f62a765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:53.194012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cbf3fe-acad-404a-a4c9-47296f62a765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:53.195548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cbf3fe-acad-404a-a4c9-47296f62a765 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:53.205728 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 731/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:13:55.441550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5bf1-0c69-48d9-ae0d-1662d1b3c59e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:55.442027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5bf1-0c69-48d9-ae0d-1662d1b3c59e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:55.442140 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5bf1-0c69-48d9-ae0d-1662d1b3c59e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:55.443575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:55.448963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b98dd-0e68-410d-b683-e823cb37d27a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:55.449405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b98dd-0e68-410d-b683-e823cb37d27a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:55.449511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b98dd-0e68-410d-b683-e823cb37d27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:55.467462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-449b98dd-0e68-410d-b683-e823cb37d27a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:55.468648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-449b98dd-0e68-410d-b683-e823cb37d27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:55.476381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:55.512810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc5bf1-0c69-48d9-ae0d-1662d1b3c59e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:55.523945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc5bf1-0c69-48d9-ae0d-1662d1b3c59e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:55.527460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd7dd6-2cdf-438b-a1b6-a6a66ff38aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:55.528092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd7dd6-2cdf-438b-a1b6-a6a66ff38aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:55.528379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd7dd6-2cdf-438b-a1b6-a6a66ff38aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:55.528656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd7dd6-2cdf-438b-a1b6-a6a66ff38aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:55.537961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd7dd6-2cdf-438b-a1b6-a6a66ff38aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:55.539410 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd7dd6-2cdf-438b-a1b6-a6a66ff38aa9 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:55.549669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 735/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:57.788903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863fcdc-395c-4c07-9384-821c7116c346 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:57.789459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863fcdc-395c-4c07-9384-821c7116c346 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:57.789459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863fcdc-395c-4c07-9384-821c7116c346 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:57.790769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:13:57.796576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86ba62-760e-42bf-96ef-0428f9a55669 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:57.797095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86ba62-760e-42bf-96ef-0428f9a55669 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:57.797200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86ba62-760e-42bf-96ef-0428f9a55669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:57.815008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8b86ba62-760e-42bf-96ef-0428f9a55669 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:57.816230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8b86ba62-760e-42bf-96ef-0428f9a55669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:57.823976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:13:57.859991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863fcdc-395c-4c07-9384-821c7116c346 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:57.870489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863fcdc-395c-4c07-9384-821c7116c346 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:57.873924 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b775a8-fe7b-461f-aedb-bbbcf0285898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:57.875162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b775a8-fe7b-461f-aedb-bbbcf0285898 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:57.875437 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b775a8-fe7b-461f-aedb-bbbcf0285898 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:57.875705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b775a8-fe7b-461f-aedb-bbbcf0285898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:57.884809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b775a8-fe7b-461f-aedb-bbbcf0285898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:57.886279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b775a8-fe7b-461f-aedb-bbbcf0285898 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:57.895877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 734/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:14:07.809436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=17 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:07.891725 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=17 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:31.756528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34f96c-ac02-4d5c-abdf-d9865e1e56f7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:31.757121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34f96c-ac02-4d5c-abdf-d9865e1e56f7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:31.757121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34f96c-ac02-4d5c-abdf-d9865e1e56f7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:31.758457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 737/1471] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:14:31.763693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76683c2-8046-46a7-8b80-e13304ad5722 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:31.764672 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76683c2-8046-46a7-8b80-e13304ad5722 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:31.765182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76683c2-8046-46a7-8b80-e13304ad5722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:31.782982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d76683c2-8046-46a7-8b80-e13304ad5722 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:31.784168 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d76683c2-8046-46a7-8b80-e13304ad5722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:31.792113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 735/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:14:31.839468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faabe5-e854-46c6-a2fa-91f9b2957f50 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:31.839757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faabe5-e854-46c6-a2fa-91f9b2957f50 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:31.840029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faabe5-e854-46c6-a2fa-91f9b2957f50 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:31.841383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 738/1473] 10.222.0.42 () {54 vars in 962 bytes} [Wed May 25 16:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:14:31.846687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6c7cf-f0aa-4797-9f23-7c16ac3eb5a4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:31.846981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6c7cf-f0aa-4797-9f23-7c16ac3eb5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:31.847264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6c7cf-f0aa-4797-9f23-7c16ac3eb5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:31.864886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-71b6c7cf-f0aa-4797-9f23-7c16ac3eb5a4 None None] MFA Rules not processed for user `8386ca93fe48460ba5b6f0c10dc51122`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:31.866115 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-71b6c7cf-f0aa-4797-9f23-7c16ac3eb5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:31.872488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 736/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Wed May 25 16:14:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:31.879960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64faabe5-e854-46c6-a2fa-91f9b2957f50 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:31.892070 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64faabe5-e854-46c6-a2fa-91f9b2957f50 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:31.895622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfdd2f-c34b-4a4c-b950-8c4934182ee8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8386ca93fe48460ba5b6f0c10dc51122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:31.896248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfdd2f-c34b-4a4c-b950-8c4934182ee8 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:31.896535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfdd2f-c34b-4a4c-b950-8c4934182ee8 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:31.896839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfdd2f-c34b-4a4c-b950-8c4934182ee8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:31.905868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfdd2f-c34b-4a4c-b950-8c4934182ee8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:31.907299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfdd2f-c34b-4a4c-b950-8c4934182ee8 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:31.916747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 739/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:14:34.226518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c578b7-40a5-44f0-9cd3-b9abbfb47bd6 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:34.227085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c578b7-40a5-44f0-9cd3-b9abbfb47bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:34.227085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c578b7-40a5-44f0-9cd3-b9abbfb47bd6 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:34.228458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 737/1476] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:14:34.234027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2766723-e175-43f6-aff6-39dbd74a5414 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:34.234487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2766723-e175-43f6-aff6-39dbd74a5414 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:34.234597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2766723-e175-43f6-aff6-39dbd74a5414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:34.252332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d2766723-e175-43f6-aff6-39dbd74a5414 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:34.253512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d2766723-e175-43f6-aff6-39dbd74a5414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:34.261208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 740/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:34.284890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c578b7-40a5-44f0-9cd3-b9abbfb47bd6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:34.298130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c578b7-40a5-44f0-9cd3-b9abbfb47bd6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:34.302607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a7e6f-8a34-44cd-b553-5deba95538a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:34.303251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a7e6f-8a34-44cd-b553-5deba95538a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:34.303537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a7e6f-8a34-44cd-b553-5deba95538a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:34.303816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a7e6f-8a34-44cd-b553-5deba95538a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:34.304539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a7e6f-8a34-44cd-b553-5deba95538a3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:34.306364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-db0a7e6f-8a34-44cd-b553-5deba95538a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:14:34.306364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 25 16:14:34.306364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: May 25 16:14:34.307531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a7e6f-8a34-44cd-b553-5deba95538a3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:34.327397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 738/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Wed May 25 16:14:34 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:14:36.560106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03f24f-30aa-4134-9336-108ecde05490 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:36.561235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03f24f-30aa-4134-9336-108ecde05490 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:36.561235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03f24f-30aa-4134-9336-108ecde05490 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:36.562770 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 741/1479] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:14:36.568025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b8f13-d2a5-4b5b-9c18-5009ee62134f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:36.568462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b8f13-d2a5-4b5b-9c18-5009ee62134f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:36.568587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b8f13-d2a5-4b5b-9c18-5009ee62134f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:36.587383 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4f6b8f13-d2a5-4b5b-9c18-5009ee62134f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:36.588605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4f6b8f13-d2a5-4b5b-9c18-5009ee62134f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:36.596381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 739/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:36.641230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03f24f-30aa-4134-9336-108ecde05490 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:36.656611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03f24f-30aa-4134-9336-108ecde05490 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:36.662170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4154c-7c8d-449b-bd5d-1075fc995d4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:36.662803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4154c-7c8d-449b-bd5d-1075fc995d4a service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:36.663087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4154c-7c8d-449b-bd5d-1075fc995d4a service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:36.663364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4154c-7c8d-449b-bd5d-1075fc995d4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:36.671703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4154c-7c8d-449b-bd5d-1075fc995d4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:36.671703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4154c-7c8d-449b-bd5d-1075fc995d4a service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:36.683402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 742/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:14:39.100019 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4549a-1b6a-4938-9934-529f83d77839 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:39.100596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4549a-1b6a-4938-9934-529f83d77839 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:39.100596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4549a-1b6a-4938-9934-529f83d77839 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:39.101997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 740/1482] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:14:39.107899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeef8f-76da-4d16-b66e-ff30bb4e9cef None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:39.108306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeef8f-76da-4d16-b66e-ff30bb4e9cef None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:39.108634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeef8f-76da-4d16-b66e-ff30bb4e9cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:39.127513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cfeeef8f-76da-4d16-b66e-ff30bb4e9cef None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:39.128697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cfeeef8f-76da-4d16-b66e-ff30bb4e9cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:39.136479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 743/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:39.174987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4549a-1b6a-4938-9934-529f83d77839 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:39.186145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4549a-1b6a-4938-9934-529f83d77839 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:39.189890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc386ce-d971-4694-bf4e-5f0bb75603bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:39.190521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc386ce-d971-4694-bf4e-5f0bb75603bf service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:39.190803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc386ce-d971-4694-bf4e-5f0bb75603bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:39.191078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc386ce-d971-4694-bf4e-5f0bb75603bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:39.200638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc386ce-d971-4694-bf4e-5f0bb75603bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:39.202877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc386ce-d971-4694-bf4e-5f0bb75603bf service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:39.212998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 741/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:14:41.512508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e7ab-adb3-459b-941e-2cbd67fd5d26 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:41.513080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e7ab-adb3-459b-941e-2cbd67fd5d26 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:41.513080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e7ab-adb3-459b-941e-2cbd67fd5d26 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:41.515325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 744/1485] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:14:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:14:41.521280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037ab57-fc27-4a2b-90e9-5c82442ddfa8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:41.521784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037ab57-fc27-4a2b-90e9-5c82442ddfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:41.521914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037ab57-fc27-4a2b-90e9-5c82442ddfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:41.541461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a037ab57-fc27-4a2b-90e9-5c82442ddfa8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:41.542697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a037ab57-fc27-4a2b-90e9-5c82442ddfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:41.551365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 742/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:14:41.599496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3e7ab-adb3-459b-941e-2cbd67fd5d26 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:41.610382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3e7ab-adb3-459b-941e-2cbd67fd5d26 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:41.614135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72642e8-8229-4dcf-aeff-52e0339cc8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:41.614780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72642e8-8229-4dcf-aeff-52e0339cc8ce service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:41.615064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72642e8-8229-4dcf-aeff-52e0339cc8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:41.615348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72642e8-8229-4dcf-aeff-52e0339cc8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:41.625018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72642e8-8229-4dcf-aeff-52e0339cc8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:41.626551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72642e8-8229-4dcf-aeff-52e0339cc8ce service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:41.636834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 745/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:14:43.963041 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abbba7-5004-4142-8037-0fdf5bce96a4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:43.963641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abbba7-5004-4142-8037-0fdf5bce96a4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:43.963641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abbba7-5004-4142-8037-0fdf5bce96a4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:43.964991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 743/1488] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:14:43.970811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2eb8c1-9dae-4293-8f14-50b1f4445be9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:43.971239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2eb8c1-9dae-4293-8f14-50b1f4445be9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:43.971372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2eb8c1-9dae-4293-8f14-50b1f4445be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:43.989328 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cf2eb8c1-9dae-4293-8f14-50b1f4445be9 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:43.990547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cf2eb8c1-9dae-4293-8f14-50b1f4445be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:43.998263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 746/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:14:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:44.037993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abbba7-5004-4142-8037-0fdf5bce96a4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:44.049074 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abbba7-5004-4142-8037-0fdf5bce96a4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:44.053575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eebea-3da4-49e8-94d1-c0c0bc11c03a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:44.054725 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eebea-3da4-49e8-94d1-c0c0bc11c03a service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:44.055251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eebea-3da4-49e8-94d1-c0c0bc11c03a service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:44.055766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eebea-3da4-49e8-94d1-c0c0bc11c03a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:44.066038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eebea-3da4-49e8-94d1-c0c0bc11c03a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:44.067462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eebea-3da4-49e8-94d1-c0c0bc11c03a service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:44.077066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 744/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:14:46.458871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ddb5b-3b01-4c5b-a81d-912a5a644066 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:46.459692 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ddb5b-3b01-4c5b-a81d-912a5a644066 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:46.459692 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ddb5b-3b01-4c5b-a81d-912a5a644066 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:46.460859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 747/1491] 10.222.0.42 () {54 vars in 926 bytes} [Wed May 25 16:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:14:46.467706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119ef6-81e8-484a-81f0-a3ef3a0f0f05 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:46.468415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119ef6-81e8-484a-81f0-a3ef3a0f0f05 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:46.468415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119ef6-81e8-484a-81f0-a3ef3a0f0f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:46.486316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-90119ef6-81e8-484a-81f0-a3ef3a0f0f05 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:46.487546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-90119ef6-81e8-484a-81f0-a3ef3a0f0f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:46.495286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 745/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Wed May 25 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:46.535201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ddb5b-3b01-4c5b-a81d-912a5a644066 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:46.546831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ddb5b-3b01-4c5b-a81d-912a5a644066 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:46.550377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c551b-98c2-4bac-88bf-24b854fea863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:46.551027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c551b-98c2-4bac-88bf-24b854fea863 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:46.551313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c551b-98c2-4bac-88bf-24b854fea863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:46.551590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c551b-98c2-4bac-88bf-24b854fea863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:46.561480 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c551b-98c2-4bac-88bf-24b854fea863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:46.562936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c551b-98c2-4bac-88bf-24b854fea863 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:46.572260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 748/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:14:56.480584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=33 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:56.568084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=33 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:27.062716 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b2806-4077-4c97-bc0d-d6429d98d31d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.063326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b2806-4077-4c97-bc0d-d6429d98d31d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.063326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b2806-4077-4c97-bc0d-d6429d98d31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.081364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-806b2806-4077-4c97-bc0d-d6429d98d31d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:27.082569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-806b2806-4077-4c97-bc0d-d6429d98d31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:27.090899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 746/1494] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:27.099966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f03e80-3822-4cba-b73b-8401c55b7ca2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.100458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f03e80-3822-4cba-b73b-8401c55b7ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.100561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f03e80-3822-4cba-b73b-8401c55b7ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.117875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-42f03e80-3822-4cba-b73b-8401c55b7ca2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:27.119051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-42f03e80-3822-4cba-b73b-8401c55b7ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:27.126789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 749/1495] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:27.131125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b2806-4077-4c97-bc0d-d6429d98d31d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:27.141257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b2806-4077-4c97-bc0d-d6429d98d31d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:27.145809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f96bd3-dddb-4bf0-bd64-ed83be3f2651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:27.146453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96bd3-dddb-4bf0-bd64-ed83be3f2651 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.146731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96bd3-dddb-4bf0-bd64-ed83be3f2651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.147002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96bd3-dddb-4bf0-bd64-ed83be3f2651 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.147720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f96bd3-dddb-4bf0-bd64-ed83be3f2651 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:27.148024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f96bd3-dddb-4bf0-bd64-ed83be3f2651 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:27.149025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-01f96bd3-dddb-4bf0-bd64-ed83be3f2651 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:16:27.149025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 25 16:16:27.149025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: May 25 16:16:27.149923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f96bd3-dddb-4bf0-bd64-ed83be3f2651 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:27.161947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 747/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:16:27.170442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f03e80-3822-4cba-b73b-8401c55b7ca2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:27.179496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f03e80-3822-4cba-b73b-8401c55b7ca2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:27.184279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197015a-55e8-427c-8e02-c8c0b85a1546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:27.184919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197015a-55e8-427c-8e02-c8c0b85a1546 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.185204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197015a-55e8-427c-8e02-c8c0b85a1546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.185479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197015a-55e8-427c-8e02-c8c0b85a1546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.186204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197015a-55e8-427c-8e02-c8c0b85a1546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:27.187972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-9197015a-55e8-427c-8e02-c8c0b85a1546 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:16:27.187972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 25 16:16:27.187972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: May 25 16:16:27.189172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197015a-55e8-427c-8e02-c8c0b85a1546 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:27.224014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 750/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:16:27 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:27.228057 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f96bd3-dddb-4bf0-bd64-ed83be3f2651 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:27.237784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f96bd3-dddb-4bf0-bd64-ed83be3f2651 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:27.293096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d7705-46ba-443a-afd5-7e4527441a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:27.293850 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7705-46ba-443a-afd5-7e4527441a0b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.294155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7705-46ba-443a-afd5-7e4527441a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.294432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7705-46ba-443a-afd5-7e4527441a0b admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.295124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d7705-46ba-443a-afd5-7e4527441a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:27.296136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-d23d7705-46ba-443a-afd5-7e4527441a0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:16:27.296136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 25 16:16:27.296136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: May 25 16:16:27.297365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d7705-46ba-443a-afd5-7e4527441a0b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:27.317790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 748/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:16:27 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:27.321765 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197015a-55e8-427c-8e02-c8c0b85a1546 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:27.331741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197015a-55e8-427c-8e02-c8c0b85a1546 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:27.336153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794d3bf-9dc1-4a9d-a147-c6a38f1eab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:27.336809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794d3bf-9dc1-4a9d-a147-c6a38f1eab41 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.337090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794d3bf-9dc1-4a9d-a147-c6a38f1eab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.337361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794d3bf-9dc1-4a9d-a147-c6a38f1eab41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.338105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794d3bf-9dc1-4a9d-a147-c6a38f1eab41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:27.339809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-a794d3bf-9dc1-4a9d-a147-c6a38f1eab41 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:16:27.339809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 25 16:16:27.339809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: May 25 16:16:27.340681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794d3bf-9dc1-4a9d-a147-c6a38f1eab41 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:27.352956 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 751/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:27.357582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d7705-46ba-443a-afd5-7e4527441a0b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:27.367366 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d7705-46ba-443a-afd5-7e4527441a0b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:27.372100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2b7b2-2a20-43ca-b7aa-83b4a78b44b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:27.372750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2b7b2-2a20-43ca-b7aa-83b4a78b44b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.373032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2b7b2-2a20-43ca-b7aa-83b4a78b44b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.373307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2b7b2-2a20-43ca-b7aa-83b4a78b44b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.374046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2b7b2-2a20-43ca-b7aa-83b4a78b44b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:27.375045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-a2f2b7b2-2a20-43ca-b7aa-83b4a78b44b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:16:27.375045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 25 16:16:27.375045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: May 25 16:16:27.375913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2b7b2-2a20-43ca-b7aa-83b4a78b44b9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:27.389565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 749/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:27.393769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794d3bf-9dc1-4a9d-a147-c6a38f1eab41 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:27.405113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794d3bf-9dc1-4a9d-a147-c6a38f1eab41 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:27.409869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0d230-5c5a-490f-a5af-2aa8928fd671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:27.410747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0d230-5c5a-490f-a5af-2aa8928fd671 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.411028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0d230-5c5a-490f-a5af-2aa8928fd671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.411306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0d230-5c5a-490f-a5af-2aa8928fd671 admin admin] PATH_INFO: `/v3/projects/a44c9ca8e0874ef68f923b7a1a95a192/users/bb8656e801444dc78a8db2d66cb6277d/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.426650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0d230-5c5a-490f-a5af-2aa8928fd671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44c9ca8e0874ef68f923b7a1a95a192, user_id=bb8656e801444dc78a8db2d66cb6277d, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:27.427688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-aba0d230-5c5a-490f-a5af-2aa8928fd671 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:16:27.427688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 25 16:16:27.427688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: May 25 16:16:27.429635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0d230-5c5a-490f-a5af-2aa8928fd671 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:27.442973 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 752/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:16:27 2022] PUT /identity/v3/projects/a44c9ca8e0874ef68f923b7a1a95a192/users/bb8656e801444dc78a8db2d66cb6277d/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:27.454355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f92c63-190c-4a6b-95ff-c9bab66e3955 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.454667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f92c63-190c-4a6b-95ff-c9bab66e3955 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.454992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f92c63-190c-4a6b-95ff-c9bab66e3955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.481656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a7f92c63-190c-4a6b-95ff-c9bab66e3955 None None] MFA Rules not processed for user `bb8656e801444dc78a8db2d66cb6277d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:27.482913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a7f92c63-190c-4a6b-95ff-c9bab66e3955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:27.547106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 750/1502] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:27.560691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda15f3-5423-41b8-aa54-0d6fe173c0c3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.561153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda15f3-5423-41b8-aa54-0d6fe173c0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.561429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda15f3-5423-41b8-aa54-0d6fe173c0c3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.562843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 753/1503] 10.222.0.42 () {54 vars in 963 bytes} [Wed May 25 16:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:16:27.567711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ec177-4000-4352-919f-bdc4c138a93b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.568047 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ec177-4000-4352-919f-bdc4c138a93b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.568269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ec177-4000-4352-919f-bdc4c138a93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.586972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-114ec177-4000-4352-919f-bdc4c138a93b None None] MFA Rules not processed for user `b9913b9eb502434985f84c4a11632ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:27.588142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-114ec177-4000-4352-919f-bdc4c138a93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:27.634968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:27.644898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 751/1504] 10.222.0.42 () {58 vars in 1060 bytes} [Wed May 25 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:27.655787 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda15f3-5423-41b8-aa54-0d6fe173c0c3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:27.666608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda15f3-5423-41b8-aa54-0d6fe173c0c3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:27.670365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cb56a-e141-4221-a293-022051ce4582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:27.670998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cb56a-e141-4221-a293-022051ce4582 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.671291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cb56a-e141-4221-a293-022051ce4582 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.671571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cb56a-e141-4221-a293-022051ce4582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.680053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cb56a-e141-4221-a293-022051ce4582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:27.681486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cb56a-e141-4221-a293-022051ce4582 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:27.690792 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 754/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:16:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:27.732932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052bbf2-79e9-4f38-b98d-045e28b0834c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.733220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052bbf2-79e9-4f38-b98d-045e28b0834c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.733498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052bbf2-79e9-4f38-b98d-045e28b0834c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.734826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 752/1506] 10.222.0.42 () {52 vars in 829 bytes} [Wed May 25 16:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:16:27.738337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cb56a-e141-4221-a293-022051ce4582 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:27.800325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cb56a-e141-4221-a293-022051ce4582 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:27.804839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:27.805752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.806041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.806319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] PATH_INFO: `/v3/users/bb8656e801444dc78a8db2d66cb6277d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.807532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb8656e801444dc78a8db2d66cb6277d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:27.808718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:16:27.808718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 25 16:16:27.808718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: May 25 16:16:27.810104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:27.872810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb8656e801444dc78a8db2d66cb6277d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:27.873496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb8656e801444dc78a8db2d66cb6277d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:27.874147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb8656e801444dc78a8db2d66cb6277d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:27.893203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb8656e801444dc78a8db2d66cb6277d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:27.906182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb8656e801444dc78a8db2d66cb6277d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:27.910128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:27.922331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 755/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:16:27 2022] DELETE /identity/v3/users/bb8656e801444dc78a8db2d66cb6277d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:27.926772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7040cd-9484-4f57-966e-a8b8dd161370 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:27.927062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7040cd-9484-4f57-966e-a8b8dd161370 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:27.927338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7040cd-9484-4f57-966e-a8b8dd161370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:27.944541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4d7040cd-9484-4f57-966e-a8b8dd161370 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:27.945736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4d7040cd-9484-4f57-966e-a8b8dd161370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:28.003703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 753/1508] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:28.013867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:28.015271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:28.072931 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0623a33-6d07-4ba0-95b9-61028b45c9bc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:28.076605 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2641b8-f9dc-440f-84ff-b50f53173f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:28.077254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2641b8-f9dc-440f-84ff-b50f53173f59 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:28.077541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2641b8-f9dc-440f-84ff-b50f53173f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:28.077858 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2641b8-f9dc-440f-84ff-b50f53173f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:28.086696 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2641b8-f9dc-440f-84ff-b50f53173f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:28.088143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2641b8-f9dc-440f-84ff-b50f53173f59 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:28.098014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 756/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:16:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:28.466622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7040cd-9484-4f57-966e-a8b8dd161370 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:28.468865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4d7040cd-9484-4f57-966e-a8b8dd161370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:28.490123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7040cd-9484-4f57-966e-a8b8dd161370 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:28.494875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:28.496279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:28.496562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:28.496841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] PATH_INFO: `/v3/projects/a44c9ca8e0874ef68f923b7a1a95a192` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:28.498522 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a44c9ca8e0874ef68f923b7a1a95a192)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:28.499099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:16:28.499099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 25 16:16:28.499099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: May 25 16:16:28.500492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:28.561307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a44c9ca8e0874ef68f923b7a1a95a192', 'request_id': 'req-42f96721-8732-439a-95c0-804e8a4db502'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:28.567245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a44c9ca8e0874ef68f923b7a1a95a192', 'request_id': 'req-42f96721-8732-439a-95c0-804e8a4db502'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:28.580557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a44c9ca8e0874ef68f923b7a1a95a192', 'request_id': 'req-42f96721-8732-439a-95c0-804e8a4db502'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:28.595550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] The token cache is being invalidate because project a44c9ca8e0874ef68f923b7a1a95a192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:28.595843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:28.596574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-42f96721-8732-439a-95c0-804e8a4db502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:28.598200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 754/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:16:28 2022] DELETE /identity/v3/projects/a44c9ca8e0874ef68f923b7a1a95a192 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:38.093952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:38.592076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:15.238979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaa57c-ba5e-4672-857c-9251a01fc213 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:15.239571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaa57c-ba5e-4672-857c-9251a01fc213 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:15.239571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaa57c-ba5e-4672-857c-9251a01fc213 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:15.240933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:18:15.245632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74371778-1482-4cb4-8eca-99427bf69708 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:15.246063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74371778-1482-4cb4-8eca-99427bf69708 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:15.246224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74371778-1482-4cb4-8eca-99427bf69708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:15.264126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-74371778-1482-4cb4-8eca-99427bf69708 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:15.265333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-74371778-1482-4cb4-8eca-99427bf69708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:15.323017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:15.491321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaa57c-ba5e-4672-857c-9251a01fc213 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:15.493164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-baaaa57c-ba5e-4672-857c-9251a01fc213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:15.570160 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaa57c-ba5e-4672-857c-9251a01fc213 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:15.576010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85b118-ab35-4467-9e98-24adc67b4167 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:15.576872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85b118-ab35-4467-9e98-24adc67b4167 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:15.577150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85b118-ab35-4467-9e98-24adc67b4167 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:15.577432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85b118-ab35-4467-9e98-24adc67b4167 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:15.585433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85b118-ab35-4467-9e98-24adc67b4167 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:15.587064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85b118-ab35-4467-9e98-24adc67b4167 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:15.597236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 758/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:18:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:16.491004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886bcc8-05e7-4423-9a24-747563d13e3d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:16.491511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886bcc8-05e7-4423-9a24-747563d13e3d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:16.491623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886bcc8-05e7-4423-9a24-747563d13e3d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:16.493091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:18:16.498640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e902fc-18b6-4bd1-ab69-cde04376732b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:16.499097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e902fc-18b6-4bd1-ab69-cde04376732b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:16.499212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e902fc-18b6-4bd1-ab69-cde04376732b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:16.517445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-08e902fc-18b6-4bd1-ab69-cde04376732b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:16.518738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-08e902fc-18b6-4bd1-ab69-cde04376732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:16.526544 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:16.679271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886bcc8-05e7-4423-9a24-747563d13e3d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:16.695368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886bcc8-05e7-4423-9a24-747563d13e3d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:16.701812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b5756-0f98-4144-8b42-c70886653dbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:16.702458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b5756-0f98-4144-8b42-c70886653dbf service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:16.702742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b5756-0f98-4144-8b42-c70886653dbf service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:16.703017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b5756-0f98-4144-8b42-c70886653dbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:16.711471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b5756-0f98-4144-8b42-c70886653dbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:16.713056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b5756-0f98-4144-8b42-c70886653dbf service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:16.722298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 757/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:18:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:16.848215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c7791-3180-46cd-b2d5-eef1d07a51d1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:16.848533 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c7791-3180-46cd-b2d5-eef1d07a51d1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:16.848778 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c7791-3180-46cd-b2d5-eef1d07a51d1 None None] PATH_INFO: `/v3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:16.850155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 760/1517] 10.222.0.84 () {54 vars in 925 bytes} [Wed May 25 16:18:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:18:16.856233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc294aea-69fb-46ca-956e-b77c7cd7aa70 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:16.856656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc294aea-69fb-46ca-956e-b77c7cd7aa70 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:16.856967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc294aea-69fb-46ca-956e-b77c7cd7aa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:16.875791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-dc294aea-69fb-46ca-956e-b77c7cd7aa70 None None] MFA Rules not processed for user `6719421144734ef49b3809df30a45961`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:16.876986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dc294aea-69fb-46ca-956e-b77c7cd7aa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:16.938094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 758/1518] 10.222.0.84 () {58 vars in 1015 bytes} [Wed May 25 16:18:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:20.895736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876abca-1151-421f-9dea-3cbf7180d398 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:20.896290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876abca-1151-421f-9dea-3cbf7180d398 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:20.896290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876abca-1151-421f-9dea-3cbf7180d398 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:20.897703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 761/1519] 10.222.0.42 () {54 vars in 960 bytes} [Wed May 25 16:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:18:20.902583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7975104-4393-4f64-861b-5760cd7762b8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:20.903039 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7975104-4393-4f64-861b-5760cd7762b8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:20.903147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7975104-4393-4f64-861b-5760cd7762b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:20.924209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e7975104-4393-4f64-861b-5760cd7762b8 None None] MFA Rules not processed for user `6719421144734ef49b3809df30a45961`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:20.925393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e7975104-4393-4f64-861b-5760cd7762b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:20.933752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 759/1520] 10.222.0.42 () {58 vars in 1057 bytes} [Wed May 25 16:18:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:18:20.941837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876abca-1151-421f-9dea-3cbf7180d398 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:20.955270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876abca-1151-421f-9dea-3cbf7180d398 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:20.960150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21628524-ed5f-4871-993a-850d9d65e0a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:20.960780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21628524-ed5f-4871-993a-850d9d65e0a2 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:20.961063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21628524-ed5f-4871-993a-850d9d65e0a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:20.961339 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21628524-ed5f-4871-993a-850d9d65e0a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:20.969738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21628524-ed5f-4871-993a-850d9d65e0a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:20.971361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21628524-ed5f-4871-993a-850d9d65e0a2 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:20.980925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 762/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:18:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:28.023909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff5e0-2181-4941-aa30-5fbc259a38d5 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:28.024487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff5e0-2181-4941-aa30-5fbc259a38d5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:28.024487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff5e0-2181-4941-aa30-5fbc259a38d5 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:28.026100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:18:28.031133 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add86a81-8e3d-4622-9c26-ccf3bc15bdb2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:28.031539 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add86a81-8e3d-4622-9c26-ccf3bc15bdb2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:28.031695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add86a81-8e3d-4622-9c26-ccf3bc15bdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:28.049568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-add86a81-8e3d-4622-9c26-ccf3bc15bdb2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:28.050775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-add86a81-8e3d-4622-9c26-ccf3bc15bdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:28.058484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:28.211932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ff5e0-2181-4941-aa30-5fbc259a38d5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:28.226754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ff5e0-2181-4941-aa30-5fbc259a38d5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:28.232735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729cd63-1f6e-4eeb-94c5-941faa0504de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:28.233370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729cd63-1f6e-4eeb-94c5-941faa0504de service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:28.233653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729cd63-1f6e-4eeb-94c5-941faa0504de service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:28.233959 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729cd63-1f6e-4eeb-94c5-941faa0504de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:28.241911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729cd63-1f6e-4eeb-94c5-941faa0504de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:28.243491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729cd63-1f6e-4eeb-94c5-941faa0504de service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:28.252675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 761/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:29.139007 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ca773-71c1-4b03-9415-074dae047133 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:29.139586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ca773-71c1-4b03-9415-074dae047133 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:29.139586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ca773-71c1-4b03-9415-074dae047133 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:29.140970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:18:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:18:29.145902 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32039d32-e2df-4eb4-bc3d-7aa1558d55d7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:29.146373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32039d32-e2df-4eb4-bc3d-7aa1558d55d7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:29.146473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32039d32-e2df-4eb4-bc3d-7aa1558d55d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:29.164847 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-32039d32-e2df-4eb4-bc3d-7aa1558d55d7 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:29.166085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-32039d32-e2df-4eb4-bc3d-7aa1558d55d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:29.173871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:29.331836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ca773-71c1-4b03-9415-074dae047133 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:29.346635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ca773-71c1-4b03-9415-074dae047133 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:29.351936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d168d-ba74-4671-b3a2-db51254f7e83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:29.352566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d168d-ba74-4671-b3a2-db51254f7e83 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:29.352849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d168d-ba74-4671-b3a2-db51254f7e83 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:29.353123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d168d-ba74-4671-b3a2-db51254f7e83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:29.361273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d168d-ba74-4671-b3a2-db51254f7e83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:29.362884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d168d-ba74-4671-b3a2-db51254f7e83 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:29.372122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 765/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:18:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:33.879515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e90a2f-aec8-474f-97b3-f9bba4f34c1f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:33.880080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e90a2f-aec8-474f-97b3-f9bba4f34c1f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:33.880080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e90a2f-aec8-474f-97b3-f9bba4f34c1f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:33.881463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:18:33.886795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3e298-2a2f-48a9-87df-b7e907cbad19 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:33.887244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3e298-2a2f-48a9-87df-b7e907cbad19 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:33.887352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3e298-2a2f-48a9-87df-b7e907cbad19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:33.905495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1ed3e298-2a2f-48a9-87df-b7e907cbad19 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:33.906725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1ed3e298-2a2f-48a9-87df-b7e907cbad19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:33.914732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:33.952180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e90a2f-aec8-474f-97b3-f9bba4f34c1f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:33.954031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-99e90a2f-aec8-474f-97b3-f9bba4f34c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:34.013531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e90a2f-aec8-474f-97b3-f9bba4f34c1f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:34.017071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b446b99-3a9d-42e0-8984-0213a1d6a293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:34.017699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b446b99-3a9d-42e0-8984-0213a1d6a293 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:34.018012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b446b99-3a9d-42e0-8984-0213a1d6a293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:34.018287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b446b99-3a9d-42e0-8984-0213a1d6a293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:34.032093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b446b99-3a9d-42e0-8984-0213a1d6a293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:34.033511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b446b99-3a9d-42e0-8984-0213a1d6a293 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:34.038628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:34.044624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 764/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:36.264189 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5061f-616f-4e13-8323-c079b172f1f0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:36.264762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5061f-616f-4e13-8323-c079b172f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:36.264762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5061f-616f-4e13-8323-c079b172f1f0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:36.266167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:18:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:18:36.271435 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc00c1-6ae9-4aa4-984f-e99a8b5d7592 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:36.271847 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc00c1-6ae9-4aa4-984f-e99a8b5d7592 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:36.272023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc00c1-6ae9-4aa4-984f-e99a8b5d7592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:36.289887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4ebc00c1-6ae9-4aa4-984f-e99a8b5d7592 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:36.291060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4ebc00c1-6ae9-4aa4-984f-e99a8b5d7592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:36.299549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:36.335626 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5061f-616f-4e13-8323-c079b172f1f0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:36.346251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5061f-616f-4e13-8323-c079b172f1f0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:36.349797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372f8a4-3555-47ae-b1b1-fda370fe8a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:36.350433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372f8a4-3555-47ae-b1b1-fda370fe8a19 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:36.350717 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372f8a4-3555-47ae-b1b1-fda370fe8a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:36.350993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372f8a4-3555-47ae-b1b1-fda370fe8a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:36.359886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372f8a4-3555-47ae-b1b1-fda370fe8a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:36.361351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372f8a4-3555-47ae-b1b1-fda370fe8a19 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:36.374420 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 768/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:40.292055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b6ec-7e8f-43ee-bd2d-664852a5d075 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:40.292633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b6ec-7e8f-43ee-bd2d-664852a5d075 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:40.292633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b6ec-7e8f-43ee-bd2d-664852a5d075 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:40.294572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:18:40.299965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c0c2e-17f2-4dd7-9d6e-465385a7039b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:40.300378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c0c2e-17f2-4dd7-9d6e-465385a7039b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:40.300525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c0c2e-17f2-4dd7-9d6e-465385a7039b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:40.306824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=62 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:40.320203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-488c0c2e-17f2-4dd7-9d6e-465385a7039b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:40.321405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-488c0c2e-17f2-4dd7-9d6e-465385a7039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:40.329122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:40.603004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181b6ec-7e8f-43ee-bd2d-664852a5d075 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:40.604694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f181b6ec-7e8f-43ee-bd2d-664852a5d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:40.662239 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181b6ec-7e8f-43ee-bd2d-664852a5d075 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:40.665982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e038e90-8f78-4187-90a2-5de21c671dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:40.666617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038e90-8f78-4187-90a2-5de21c671dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:40.666898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038e90-8f78-4187-90a2-5de21c671dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:40.667169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038e90-8f78-4187-90a2-5de21c671dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:40.676810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e038e90-8f78-4187-90a2-5de21c671dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:40.678256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e038e90-8f78-4187-90a2-5de21c671dc7 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:40.687719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 767/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:42.923122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0706de-9352-436c-bfd5-54b9fd958f6f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:42.923717 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0706de-9352-436c-bfd5-54b9fd958f6f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:42.923717 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0706de-9352-436c-bfd5-54b9fd958f6f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:42.925077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:18:42.930302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870d717-e30d-4eee-bfe5-3a9d43e7f7bb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:42.930701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870d717-e30d-4eee-bfe5-3a9d43e7f7bb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:42.930863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870d717-e30d-4eee-bfe5-3a9d43e7f7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:42.949097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3870d717-e30d-4eee-bfe5-3a9d43e7f7bb None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:42.951284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3870d717-e30d-4eee-bfe5-3a9d43e7f7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:42.959062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:43.227440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0706de-9352-436c-bfd5-54b9fd958f6f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:43.229097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9d0706de-9352-436c-bfd5-54b9fd958f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:43.242955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0706de-9352-436c-bfd5-54b9fd958f6f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:43.246586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918e323-35b7-4041-950c-86b3fcb53d57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:43.247357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918e323-35b7-4041-950c-86b3fcb53d57 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:43.247639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918e323-35b7-4041-950c-86b3fcb53d57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:43.247914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918e323-35b7-4041-950c-86b3fcb53d57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:43.256664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918e323-35b7-4041-950c-86b3fcb53d57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:43.258124 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918e323-35b7-4041-950c-86b3fcb53d57 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:43.267883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 771/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:18:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:52.943863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=32 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:53.264136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:15.353708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22245835-6bf5-4b23-8242-6cd3862961d2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.355243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22245835-6bf5-4b23-8242-6cd3862961d2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.355754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22245835-6bf5-4b23-8242-6cd3862961d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.387844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-22245835-6bf5-4b23-8242-6cd3862961d2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.389345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-22245835-6bf5-4b23-8242-6cd3862961d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.402591 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 769/1540] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:15.414654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7880-ac88-4320-80ff-7edb44a9afe4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.415251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7880-ac88-4320-80ff-7edb44a9afe4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.415251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7880-ac88-4320-80ff-7edb44a9afe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.459037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-96ac7880-ac88-4320-80ff-7edb44a9afe4 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.460309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-96ac7880-ac88-4320-80ff-7edb44a9afe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.474231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 772/1541] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:15.480291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22245835-6bf5-4b23-8242-6cd3862961d2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:15.496633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22245835-6bf5-4b23-8242-6cd3862961d2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:15.507312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad023a-85df-4730-a644-88021517c118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:15.508203 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad023a-85df-4730-a644-88021517c118 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.508768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad023a-85df-4730-a644-88021517c118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.509214 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad023a-85df-4730-a644-88021517c118 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.510561 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad023a-85df-4730-a644-88021517c118 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:15.513843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad023a-85df-4730-a644-88021517c118 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:15.515843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad023a-85df-4730-a644-88021517c118 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:15.533343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 770/1542] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:15.543088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac7880-ac88-4320-80ff-7edb44a9afe4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:15.558248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac7880-ac88-4320-80ff-7edb44a9afe4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:15.562807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7aa77-2d4e-45f8-838b-f04bd53b6679 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.563306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7aa77-2d4e-45f8-838b-f04bd53b6679 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.563429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff662d-824f-4a01-96a4-245062d8f9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:15.563876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7aa77-2d4e-45f8-838b-f04bd53b6679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.564015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff662d-824f-4a01-96a4-245062d8f9f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.564265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff662d-824f-4a01-96a4-245062d8f9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.564550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff662d-824f-4a01-96a4-245062d8f9f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.565251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff662d-824f-4a01-96a4-245062d8f9f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:15.567327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff662d-824f-4a01-96a4-245062d8f9f0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:15.594485 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-02a7aa77-2d4e-45f8-838b-f04bd53b6679 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.596509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-02a7aa77-2d4e-45f8-838b-f04bd53b6679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.608679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 773/1543] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:15.614865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff662d-824f-4a01-96a4-245062d8f9f0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:15.618754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 771/1544] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:20:15.630642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c3dc3-b1aa-49ae-a713-5832f013de8a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.631640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c3dc3-b1aa-49ae-a713-5832f013de8a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.632244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c3dc3-b1aa-49ae-a713-5832f013de8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.668505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-912c3dc3-b1aa-49ae-a713-5832f013de8a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.670012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-912c3dc3-b1aa-49ae-a713-5832f013de8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.716539 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff662d-824f-4a01-96a4-245062d8f9f0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:15.721876 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706075e7-30e4-4496-b7dc-20ff2da8242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:15.722865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706075e7-30e4-4496-b7dc-20ff2da8242e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.723327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706075e7-30e4-4496-b7dc-20ff2da8242e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.723776 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706075e7-30e4-4496-b7dc-20ff2da8242e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.724648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706075e7-30e4-4496-b7dc-20ff2da8242e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:15.730105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-706075e7-30e4-4496-b7dc-20ff2da8242e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:20:15.730105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 25 16:20:15.730105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: May 25 16:20:15.731919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706075e7-30e4-4496-b7dc-20ff2da8242e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:15.768142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 774/1545] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:15.772233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b63fd-bc04-4e90-b373-56703fb5a765 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.772808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b63fd-bc04-4e90-b373-56703fb5a765 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.773318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b63fd-bc04-4e90-b373-56703fb5a765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.794864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 772/1546] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:15.796253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-df2b63fd-bc04-4e90-b373-56703fb5a765 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.796456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3448406-6db3-4900-b4a5-0991e82910dc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.796935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3448406-6db3-4900-b4a5-0991e82910dc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.797375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3448406-6db3-4900-b4a5-0991e82910dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.798029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-df2b63fd-bc04-4e90-b373-56703fb5a765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.808304 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 775/1547] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:15.809759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b63fd-bc04-4e90-b373-56703fb5a765 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:15.824081 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a3448406-6db3-4900-b4a5-0991e82910dc None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.825595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a3448406-6db3-4900-b4a5-0991e82910dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.829218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b63fd-bc04-4e90-b373-56703fb5a765 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:15.834477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25a8a5-9473-4207-a22e-32b591040c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:15.834759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 773/1548] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:15.835217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25a8a5-9473-4207-a22e-32b591040c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.835615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25a8a5-9473-4207-a22e-32b591040c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.835942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25a8a5-9473-4207-a22e-32b591040c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.836741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25a8a5-9473-4207-a22e-32b591040c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:15.837456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62db0c-c6b6-4620-b37d-e224eb5239df None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.837958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62db0c-c6b6-4620-b37d-e224eb5239df None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.838589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62db0c-c6b6-4620-b37d-e224eb5239df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.838881 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25a8a5-9473-4207-a22e-32b591040c84 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:15.853767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 776/1549] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:15.859207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f30c67-72fa-48dd-8fc3-08e1f2d617d7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.859729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f30c67-72fa-48dd-8fc3-08e1f2d617d7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.860238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f30c67-72fa-48dd-8fc3-08e1f2d617d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.861684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bd62db0c-c6b6-4620-b37d-e224eb5239df None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.863083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bd62db0c-c6b6-4620-b37d-e224eb5239df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.873746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 774/1550] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:15.874777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62db0c-c6b6-4620-b37d-e224eb5239df None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:15.882178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d4f30c67-72fa-48dd-8fc3-08e1f2d617d7 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.883845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d4f30c67-72fa-48dd-8fc3-08e1f2d617d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.887096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62db0c-c6b6-4620-b37d-e224eb5239df None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:15.894700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 777/1551] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:15.896141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de8258-cbc2-4996-a43f-0d700cbbe6d5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.896514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de8258-cbc2-4996-a43f-0d700cbbe6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.896868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de8258-cbc2-4996-a43f-0d700cbbe6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.898421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e4d82-1e18-4eac-9678-acdbf61e9931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:15.899327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e4d82-1e18-4eac-9678-acdbf61e9931 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.899790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e4d82-1e18-4eac-9678-acdbf61e9931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.900251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e4d82-1e18-4eac-9678-acdbf61e9931 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.902074 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e4d82-1e18-4eac-9678-acdbf61e9931 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:15.902635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e4d82-1e18-4eac-9678-acdbf61e9931 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:15.904791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e4d82-1e18-4eac-9678-acdbf61e9931 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:15.916764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f2de8258-cbc2-4996-a43f-0d700cbbe6d5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.918072 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f2de8258-cbc2-4996-a43f-0d700cbbe6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.918599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 775/1552] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:15.919717 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0053e-3da8-4ebd-bd96-281a0e9c2709 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.920003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0053e-3da8-4ebd-bd96-281a0e9c2709 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.920278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0053e-3da8-4ebd-bd96-281a0e9c2709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.931682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 778/1553] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:15.933172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de8258-cbc2-4996-a43f-0d700cbbe6d5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:15.942702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de8258-cbc2-4996-a43f-0d700cbbe6d5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:15.947018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-dcb0053e-3da8-4ebd-bd96-281a0e9c2709 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.947291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec53fa-af96-467a-ac8f-f7829b0d3e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:15.948597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec53fa-af96-467a-ac8f-f7829b0d3e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.948880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec53fa-af96-467a-ac8f-f7829b0d3e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.949153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec53fa-af96-467a-ac8f-f7829b0d3e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.949273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dcb0053e-3da8-4ebd-bd96-281a0e9c2709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.949910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec53fa-af96-467a-ac8f-f7829b0d3e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:15.951505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec53fa-af96-467a-ac8f-f7829b0d3e90 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:15.957089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 776/1554] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:15.960706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24320863-f8ed-497a-8653-e0ed3fb6621e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.960706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24320863-f8ed-497a-8653-e0ed3fb6621e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.960706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24320863-f8ed-497a-8653-e0ed3fb6621e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.961613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 779/1555] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:15.962774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823f96f-34bb-45e2-ad46-7d1c6c6e8318 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.963067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823f96f-34bb-45e2-ad46-7d1c6c6e8318 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.963378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823f96f-34bb-45e2-ad46-7d1c6c6e8318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.980390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-24320863-f8ed-497a-8653-e0ed3fb6621e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.981489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3823f96f-34bb-45e2-ad46-7d1c6c6e8318 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:15.981628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-24320863-f8ed-497a-8653-e0ed3fb6621e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.982688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3823f96f-34bb-45e2-ad46-7d1c6c6e8318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:15.992863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 777/1556] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:15.993033 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 780/1557] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:15.993143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dcfc4-c5a7-4fca-a5d5-7b9ef67ece2d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:15.993143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dcfc4-c5a7-4fca-a5d5-7b9ef67ece2d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:15.993143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dcfc4-c5a7-4fca-a5d5-7b9ef67ece2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:15.993419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823f96f-34bb-45e2-ad46-7d1c6c6e8318 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.004577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823f96f-34bb-45e2-ad46-7d1c6c6e8318 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.009369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f29fb-e0d4-4e4e-89fa-8655a778920d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.010108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f29fb-e0d4-4e4e-89fa-8655a778920d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.011276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f29fb-e0d4-4e4e-89fa-8655a778920d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.011562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f29fb-e0d4-4e4e-89fa-8655a778920d admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.012181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ab8dcfc4-c5a7-4fca-a5d5-7b9ef67ece2d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:16.012418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f29fb-e0d4-4e4e-89fa-8655a778920d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.013370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ab8dcfc4-c5a7-4fca-a5d5-7b9ef67ece2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:16.014418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f29fb-e0d4-4e4e-89fa-8655a778920d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.021782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 778/1558] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:16.024664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9006e1f-9a06-4f24-a7fa-00e8208507ff None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.024664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9006e1f-9a06-4f24-a7fa-00e8208507ff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.024664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9006e1f-9a06-4f24-a7fa-00e8208507ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.046226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 781/1559] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:15 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.047348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f29fb-e0d4-4e4e-89fa-8655a778920d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.048341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d9006e1f-9a06-4f24-a7fa-00e8208507ff None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:16.050039 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d9006e1f-9a06-4f24-a7fa-00e8208507ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:16.099743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:16.112241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f29fb-e0d4-4e4e-89fa-8655a778920d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.112417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 779/1560] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:16.113093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9006e1f-9a06-4f24-a7fa-00e8208507ff None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.123147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9006e1f-9a06-4f24-a7fa-00e8208507ff None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.123937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396ba4-3d52-41fe-8f34-7de7270966e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.124611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396ba4-3d52-41fe-8f34-7de7270966e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.124895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396ba4-3d52-41fe-8f34-7de7270966e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.125169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396ba4-3d52-41fe-8f34-7de7270966e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.125930 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d396ba4-3d52-41fe-8f34-7de7270966e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:16.126241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d396ba4-3d52-41fe-8f34-7de7270966e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.127258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-0d396ba4-3d52-41fe-8f34-7de7270966e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:20:16.127258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 25 16:20:16.127258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: May 25 16:20:16.127745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27692230-7b7d-4e5b-845a-3a2d651125b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.128164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d396ba4-3d52-41fe-8f34-7de7270966e3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.128419 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27692230-7b7d-4e5b-845a-3a2d651125b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.128703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27692230-7b7d-4e5b-845a-3a2d651125b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.128978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27692230-7b7d-4e5b-845a-3a2d651125b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.129701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27692230-7b7d-4e5b-845a-3a2d651125b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:16.130037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27692230-7b7d-4e5b-845a-3a2d651125b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.131640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27692230-7b7d-4e5b-845a-3a2d651125b4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.142054 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 782/1561] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:16.142654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396ba4-3d52-41fe-8f34-7de7270966e3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.151043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 780/1562] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:16.151043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27692230-7b7d-4e5b-845a-3a2d651125b4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.158610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396ba4-3d52-41fe-8f34-7de7270966e3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.159444 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27692230-7b7d-4e5b-845a-3a2d651125b4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.163659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db3a09-7f3d-48bf-92e8-0c2c1b8f2af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.164241 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b554ce-af4c-4ac6-a3b7-3e3f6cc2ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.164535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3a09-7f3d-48bf-92e8-0c2c1b8f2af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.164822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3a09-7f3d-48bf-92e8-0c2c1b8f2af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.164920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b554ce-af4c-4ac6-a3b7-3e3f6cc2ec3a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.165103 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3a09-7f3d-48bf-92e8-0c2c1b8f2af5 admin admin] PATH_INFO: `/v3/projects/9d234edace28441282f594f0e03cacc5/users/b781c4c6cb61439fb781988a7da028ff/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.165206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b554ce-af4c-4ac6-a3b7-3e3f6cc2ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.165405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b554ce-af4c-4ac6-a3b7-3e3f6cc2ec3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.166143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b554ce-af4c-4ac6-a3b7-3e3f6cc2ec3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:16.166476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b554ce-af4c-4ac6-a3b7-3e3f6cc2ec3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.168053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b554ce-af4c-4ac6-a3b7-3e3f6cc2ec3a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.179794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 781/1563] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:16.180728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db3a09-7f3d-48bf-92e8-0c2c1b8f2af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d234edace28441282f594f0e03cacc5, user_id=b781c4c6cb61439fb781988a7da028ff, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.180989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80951298-28a9-4aba-a848-19b9141f5aad None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.181281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80951298-28a9-4aba-a848-19b9141f5aad None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.181552 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80951298-28a9-4aba-a848-19b9141f5aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.184105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db3a09-7f3d-48bf-92e8-0c2c1b8f2af5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.194116 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:16.200559 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 783/1564] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:16 2022] PUT /identity/v3/projects/9d234edace28441282f594f0e03cacc5/users/b781c4c6cb61439fb781988a7da028ff/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:16.201336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db3a09-7f3d-48bf-92e8-0c2c1b8f2af5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.207630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-80951298-28a9-4aba-a848-19b9141f5aad None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:16.208828 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-80951298-28a9-4aba-a848-19b9141f5aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:16.268172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db3a09-7f3d-48bf-92e8-0c2c1b8f2af5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.272791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912735e-3df5-49fe-b0ee-cc701e8fb7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.273452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912735e-3df5-49fe-b0ee-cc701e8fb7ca admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.273830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912735e-3df5-49fe-b0ee-cc701e8fb7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.274114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912735e-3df5-49fe-b0ee-cc701e8fb7ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.274624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 782/1565] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:16.278455 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912735e-3df5-49fe-b0ee-cc701e8fb7ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:16.278455 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912735e-3df5-49fe-b0ee-cc701e8fb7ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.278673 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80951298-28a9-4aba-a848-19b9141f5aad None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.278780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912735e-3df5-49fe-b0ee-cc701e8fb7ca admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.285625 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80951298-28a9-4aba-a848-19b9141f5aad None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.297259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 784/1566] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:16.297447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534761d5-1e3a-4204-9e71-d11fba48c930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.297571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bac14c-8165-4e06-9359-38ec2dcb9a27 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.297571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bac14c-8165-4e06-9359-38ec2dcb9a27 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.297571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bac14c-8165-4e06-9359-38ec2dcb9a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.297838 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534761d5-1e3a-4204-9e71-d11fba48c930 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.297838 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534761d5-1e3a-4204-9e71-d11fba48c930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.297838 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534761d5-1e3a-4204-9e71-d11fba48c930 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.297838 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534761d5-1e3a-4204-9e71-d11fba48c930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.297838 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534761d5-1e3a-4204-9e71-d11fba48c930 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.311971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c7bac14c-8165-4e06-9359-38ec2dcb9a27 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:16.313189 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c7bac14c-8165-4e06-9359-38ec2dcb9a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:16.315576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 783/1567] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/users => generated 291 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.316459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534761d5-1e3a-4204-9e71-d11fba48c930 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.324446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 785/1568] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:16.325219 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bac14c-8165-4e06-9359-38ec2dcb9a27 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.331997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534761d5-1e3a-4204-9e71-d11fba48c930 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.334326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bac14c-8165-4e06-9359-38ec2dcb9a27 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.336553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7d46b-a07a-4ca9-bd85-812df8e51658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.337319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7d46b-a07a-4ca9-bd85-812df8e51658 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.337610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7d46b-a07a-4ca9-bd85-812df8e51658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.337919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7d46b-a07a-4ca9-bd85-812df8e51658 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.338901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd725ca-b19d-4697-807c-22f4cca61d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.339046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7d46b-a07a-4ca9-bd85-812df8e51658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.339754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd725ca-b19d-4697-807c-22f4cca61d3f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.340037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd725ca-b19d-4697-807c-22f4cca61d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.340143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-e0e7d46b-a07a-4ca9-bd85-812df8e51658 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:20:16.340143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 25 16:20:16.340143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: May 25 16:20:16.340382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd725ca-b19d-4697-807c-22f4cca61d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.340972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd725ca-b19d-4697-807c-22f4cca61d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.341236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7d46b-a07a-4ca9-bd85-812df8e51658 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.342905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd725ca-b19d-4697-807c-22f4cca61d3f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.374363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 784/1569] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.374363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7d46b-a07a-4ca9-bd85-812df8e51658 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.375580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 786/1570] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.375580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb423e-2efd-4479-a0d4-ecbcdb77f45f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.375580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb423e-2efd-4479-a0d4-ecbcdb77f45f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.375580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb423e-2efd-4479-a0d4-ecbcdb77f45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.391444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c2eb423e-2efd-4479-a0d4-ecbcdb77f45f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:16.392637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c2eb423e-2efd-4479-a0d4-ecbcdb77f45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:16.440459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7d46b-a07a-4ca9-bd85-812df8e51658 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.455245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 787/1571] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:16.456700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb423e-2efd-4479-a0d4-ecbcdb77f45f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.466987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb423e-2efd-4479-a0d4-ecbcdb77f45f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.471715 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b014cb9-6bbf-4ded-ab17-0e5daee653dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.472373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b014cb9-6bbf-4ded-ab17-0e5daee653dd admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.472654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b014cb9-6bbf-4ded-ab17-0e5daee653dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.472933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b014cb9-6bbf-4ded-ab17-0e5daee653dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.473658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b014cb9-6bbf-4ded-ab17-0e5daee653dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:16.473996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b014cb9-6bbf-4ded-ab17-0e5daee653dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.475737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b014cb9-6bbf-4ded-ab17-0e5daee653dd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.487658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 788/1572] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:16.489608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b014cb9-6bbf-4ded-ab17-0e5daee653dd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.499541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b014cb9-6bbf-4ded-ab17-0e5daee653dd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.504043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967b594-2302-42fb-b49b-da84f756ba75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.504661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ab6ea-a06b-4528-a4d0-59f991a7be7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.504810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967b594-2302-42fb-b49b-da84f756ba75 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.504960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967b594-2302-42fb-b49b-da84f756ba75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.505214 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967b594-2302-42fb-b49b-da84f756ba75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.505324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ab6ea-a06b-4528-a4d0-59f991a7be7e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.505593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ab6ea-a06b-4528-a4d0-59f991a7be7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.505902 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ab6ea-a06b-4528-a4d0-59f991a7be7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.506015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967b594-2302-42fb-b49b-da84f756ba75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.506582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ab6ea-a06b-4528-a4d0-59f991a7be7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.507791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967b594-2302-42fb-b49b-da84f756ba75 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.508503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ab6ea-a06b-4528-a4d0-59f991a7be7e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.519070 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:16.524612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:16.542016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 789/1573] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.542016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967b594-2302-42fb-b49b-da84f756ba75 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.545052 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 785/1574] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.546968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ab6ea-a06b-4528-a4d0-59f991a7be7e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.615577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967b594-2302-42fb-b49b-da84f756ba75 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.627975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ab6ea-a06b-4528-a4d0-59f991a7be7e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.632595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49234469-ee6c-43f0-a67d-3d8ae544bcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.633264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49234469-ee6c-43f0-a67d-3d8ae544bcc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.633585 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49234469-ee6c-43f0-a67d-3d8ae544bcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.633869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49234469-ee6c-43f0-a67d-3d8ae544bcc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.634596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49234469-ee6c-43f0-a67d-3d8ae544bcc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:16.634905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49234469-ee6c-43f0-a67d-3d8ae544bcc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.637117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49234469-ee6c-43f0-a67d-3d8ae544bcc2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.649148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 786/1575] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:16.650010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49234469-ee6c-43f0-a67d-3d8ae544bcc2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.661101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49234469-ee6c-43f0-a67d-3d8ae544bcc2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.665967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490df9e3-8848-4abf-a649-36f64669894a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.666675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490df9e3-8848-4abf-a649-36f64669894a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.666963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490df9e3-8848-4abf-a649-36f64669894a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.667249 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490df9e3-8848-4abf-a649-36f64669894a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.667926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490df9e3-8848-4abf-a649-36f64669894a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.669871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490df9e3-8848-4abf-a649-36f64669894a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.678975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aea8be-0a1a-48f2-8655-b30e6ac53f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.679680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aea8be-0a1a-48f2-8655-b30e6ac53f5b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.679968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aea8be-0a1a-48f2-8655-b30e6ac53f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.680330 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aea8be-0a1a-48f2-8655-b30e6ac53f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.681048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aea8be-0a1a-48f2-8655-b30e6ac53f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.683847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aea8be-0a1a-48f2-8655-b30e6ac53f5b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.691154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 787/1576] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.692110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490df9e3-8848-4abf-a649-36f64669894a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.702318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 790/1577] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:16.702318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aea8be-0a1a-48f2-8655-b30e6ac53f5b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.704014 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490df9e3-8848-4abf-a649-36f64669894a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.710943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6769f69-e874-4bcd-bbd0-33a1cd366f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.711782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6769f69-e874-4bcd-bbd0-33a1cd366f4b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.712078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6769f69-e874-4bcd-bbd0-33a1cd366f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.712338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6769f69-e874-4bcd-bbd0-33a1cd366f4b admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.713071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6769f69-e874-4bcd-bbd0-33a1cd366f4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.715746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6769f69-e874-4bcd-bbd0-33a1cd366f4b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.735958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 788/1578] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/users => generated 295 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.736763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6769f69-e874-4bcd-bbd0-33a1cd366f4b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.746568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6769f69-e874-4bcd-bbd0-33a1cd366f4b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.751167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018432f2-c51a-408e-94d7-f629d86d6e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.751803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018432f2-c51a-408e-94d7-f629d86d6e4b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.752357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018432f2-c51a-408e-94d7-f629d86d6e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.752357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018432f2-c51a-408e-94d7-f629d86d6e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.753021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018432f2-c51a-408e-94d7-f629d86d6e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.754949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018432f2-c51a-408e-94d7-f629d86d6e4b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.761824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aea8be-0a1a-48f2-8655-b30e6ac53f5b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.765479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cd1ca-2456-44a5-9fcb-a430f0d9f555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.766188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cd1ca-2456-44a5-9fcb-a430f0d9f555 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.766472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cd1ca-2456-44a5-9fcb-a430f0d9f555 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.766772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cd1ca-2456-44a5-9fcb-a430f0d9f555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.776640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cd1ca-2456-44a5-9fcb-a430f0d9f555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.778288 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cd1ca-2456-44a5-9fcb-a430f0d9f555 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.785763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 789/1579] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.785763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018432f2-c51a-408e-94d7-f629d86d6e4b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.793990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 791/1580] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:16.794864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cd1ca-2456-44a5-9fcb-a430f0d9f555 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.804109 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cd1ca-2456-44a5-9fcb-a430f0d9f555 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.843767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:16.859158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018432f2-c51a-408e-94d7-f629d86d6e4b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.864807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13630d8b-503b-4264-aadd-d4bd6ab92189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.865560 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13630d8b-503b-4264-aadd-d4bd6ab92189 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.865885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13630d8b-503b-4264-aadd-d4bd6ab92189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.866165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13630d8b-503b-4264-aadd-d4bd6ab92189 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.867111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13630d8b-503b-4264-aadd-d4bd6ab92189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.868776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13630d8b-503b-4264-aadd-d4bd6ab92189 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.870487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc9383-3890-4960-b827-007b2fb60f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.871934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9383-3890-4960-b827-007b2fb60f21 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.871934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9383-3890-4960-b827-007b2fb60f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.872222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9383-3890-4960-b827-007b2fb60f21 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.872880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc9383-3890-4960-b827-007b2fb60f21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.875201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc9383-3890-4960-b827-007b2fb60f21 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.896687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 790/1581] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.896687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13630d8b-503b-4264-aadd-d4bd6ab92189 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.899862 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 792/1582] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.901445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc9383-3890-4960-b827-007b2fb60f21 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.908689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13630d8b-503b-4264-aadd-d4bd6ab92189 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.913464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc9383-3890-4960-b827-007b2fb60f21 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.915019 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553ad8b-3551-446f-b6a8-d766711649d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.915722 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ad8b-3551-446f-b6a8-d766711649d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.916007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ad8b-3551-446f-b6a8-d766711649d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.916385 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ad8b-3551-446f-b6a8-d766711649d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.917062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553ad8b-3551-446f-b6a8-d766711649d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.918970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d779b10-8e49-4351-972a-06f96e79b2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.919098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553ad8b-3551-446f-b6a8-d766711649d9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.919710 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d779b10-8e49-4351-972a-06f96e79b2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.920001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d779b10-8e49-4351-972a-06f96e79b2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.920275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d779b10-8e49-4351-972a-06f96e79b2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.921001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d779b10-8e49-4351-972a-06f96e79b2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.922938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d779b10-8e49-4351-972a-06f96e79b2f0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.934908 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 793/1583] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:16.935745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d779b10-8e49-4351-972a-06f96e79b2f0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.946702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d779b10-8e49-4351-972a-06f96e79b2f0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:16.948876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:16.952185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e9994-97ea-430b-a285-63561a0dec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:16.953215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e9994-97ea-430b-a285-63561a0dec73 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:16.953658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e9994-97ea-430b-a285-63561a0dec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:16.954134 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e9994-97ea-430b-a285-63561a0dec73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:16.955059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e9994-97ea-430b-a285-63561a0dec73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:16.956890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e9994-97ea-430b-a285-63561a0dec73 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:16.957787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 791/1584] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:16.958766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553ad8b-3551-446f-b6a8-d766711649d9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.970292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 794/1585] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:16.971349 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e9994-97ea-430b-a285-63561a0dec73 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:16.982316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e9994-97ea-430b-a285-63561a0dec73 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.048827 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553ad8b-3551-446f-b6a8-d766711649d9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.053628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf473e49-79c7-4279-871a-2db4f497b6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.054440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf473e49-79c7-4279-871a-2db4f497b6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.054728 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf473e49-79c7-4279-871a-2db4f497b6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.055004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf473e49-79c7-4279-871a-2db4f497b6c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.055766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf473e49-79c7-4279-871a-2db4f497b6c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.057016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6490-f89b-456a-8919-4fe0ba1adbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.057436 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf473e49-79c7-4279-871a-2db4f497b6c9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.057761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6490-f89b-456a-8919-4fe0ba1adbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.058258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6490-f89b-456a-8919-4fe0ba1adbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.058543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6490-f89b-456a-8919-4fe0ba1adbb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.059354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f6490-f89b-456a-8919-4fe0ba1adbb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.061486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f6490-f89b-456a-8919-4fe0ba1adbb5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.068937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 792/1586] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:17.069607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf473e49-79c7-4279-871a-2db4f497b6c9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.084278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf473e49-79c7-4279-871a-2db4f497b6c9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.090200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 795/1587] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:16 2022] POST /identity/v3/users => generated 297 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:17.090200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6490-f89b-456a-8919-4fe0ba1adbb5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.094638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4858-a887-410d-847d-b57fcd823215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.095595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4858-a887-410d-847d-b57fcd823215 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.095940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4858-a887-410d-847d-b57fcd823215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.097064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4858-a887-410d-847d-b57fcd823215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.097893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd4858-a887-410d-847d-b57fcd823215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.100005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd4858-a887-410d-847d-b57fcd823215 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.101087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6490-f89b-456a-8919-4fe0ba1adbb5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.107091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf5b1a-11e3-42ca-a822-eed7de72fe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.108482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf5b1a-11e3-42ca-a822-eed7de72fe68 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.108994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf5b1a-11e3-42ca-a822-eed7de72fe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.109762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf5b1a-11e3-42ca-a822-eed7de72fe68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.110699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf5b1a-11e3-42ca-a822-eed7de72fe68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.112571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 793/1588] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:17.112955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf5b1a-11e3-42ca-a822-eed7de72fe68 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.113465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4858-a887-410d-847d-b57fcd823215 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.123616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4858-a887-410d-847d-b57fcd823215 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.127104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 796/1589] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:17.128245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf5b1a-11e3-42ca-a822-eed7de72fe68 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.135597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26341721-011d-463c-8f21-9622dc44df36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.136285 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26341721-011d-463c-8f21-9622dc44df36 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.136646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26341721-011d-463c-8f21-9622dc44df36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.136997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26341721-011d-463c-8f21-9622dc44df36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.137841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26341721-011d-463c-8f21-9622dc44df36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.139644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26341721-011d-463c-8f21-9622dc44df36 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.139764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf5b1a-11e3-42ca-a822-eed7de72fe68 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.145467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4552037-a43d-4946-b0ef-f99df0245d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.146205 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4552037-a43d-4946-b0ef-f99df0245d9f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.146619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4552037-a43d-4946-b0ef-f99df0245d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.146958 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4552037-a43d-4946-b0ef-f99df0245d9f admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.148037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4552037-a43d-4946-b0ef-f99df0245d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.150040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4552037-a43d-4946-b0ef-f99df0245d9f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.153320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 794/1590] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:17.157507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26341721-011d-463c-8f21-9622dc44df36 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.169920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26341721-011d-463c-8f21-9622dc44df36 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.177988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 797/1591] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:17 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:17.178290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0940c0-11a3-456e-8836-ec5778f2c37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.178410 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4552037-a43d-4946-b0ef-f99df0245d9f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.178489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0940c0-11a3-456e-8836-ec5778f2c37b admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.178489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0940c0-11a3-456e-8836-ec5778f2c37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.178489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0940c0-11a3-456e-8836-ec5778f2c37b admin admin] PATH_INFO: `/v3/projects/8c0171bf802746578d0dd0267e0393d6/users/f858d1d451e44a0fbd3fdbff44220c0e/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.188623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4552037-a43d-4946-b0ef-f99df0245d9f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.194231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee974a-1a90-4da6-a326-2a658f380601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.195338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee974a-1a90-4da6-a326-2a658f380601 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.195338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee974a-1a90-4da6-a326-2a658f380601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.195536 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee974a-1a90-4da6-a326-2a658f380601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.196818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee974a-1a90-4da6-a326-2a658f380601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.199290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee974a-1a90-4da6-a326-2a658f380601 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.201408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0940c0-11a3-456e-8836-ec5778f2c37b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0171bf802746578d0dd0267e0393d6, user_id=f858d1d451e44a0fbd3fdbff44220c0e, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.202507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-1b0940c0-11a3-456e-8836-ec5778f2c37b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:20:17.202507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 25 16:20:17.202507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: May 25 16:20:17.204689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0940c0-11a3-456e-8836-ec5778f2c37b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.217316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 798/1592] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:17.218472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee974a-1a90-4da6-a326-2a658f380601 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.224889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 795/1593] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:17 2022] PUT /identity/v3/projects/8c0171bf802746578d0dd0267e0393d6/users/f858d1d451e44a0fbd3fdbff44220c0e/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:17.226995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0940c0-11a3-456e-8836-ec5778f2c37b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.230184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee974a-1a90-4da6-a326-2a658f380601 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.293969 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:17.302645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7fd38-b671-4c42-b1fe-277508d772dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.303372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7fd38-b671-4c42-b1fe-277508d772dc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.303996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7fd38-b671-4c42-b1fe-277508d772dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.304286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7fd38-b671-4c42-b1fe-277508d772dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.305315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7fd38-b671-4c42-b1fe-277508d772dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.308084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7fd38-b671-4c42-b1fe-277508d772dc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.311665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0940c0-11a3-456e-8836-ec5778f2c37b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.316790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a6ce9-bd66-4453-a4e5-1d7ce03bdc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.317663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a6ce9-bd66-4453-a4e5-1d7ce03bdc3f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.318152 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a6ce9-bd66-4453-a4e5-1d7ce03bdc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.318601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a6ce9-bd66-4453-a4e5-1d7ce03bdc3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.318942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 799/1594] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:17.319813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a6ce9-bd66-4453-a4e5-1d7ce03bdc3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.321588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a6ce9-bd66-4453-a4e5-1d7ce03bdc3f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.323398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7fd38-b671-4c42-b1fe-277508d772dc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.337260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 796/1595] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:17.337451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7fd38-b671-4c42-b1fe-277508d772dc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.337562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a6ce9-bd66-4453-a4e5-1d7ce03bdc3f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.342491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8638b1-9dde-4d15-82c4-42ee472fe337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.342803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8638b1-9dde-4d15-82c4-42ee472fe337 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.343163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8638b1-9dde-4d15-82c4-42ee472fe337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.343512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8638b1-9dde-4d15-82c4-42ee472fe337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.344324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8638b1-9dde-4d15-82c4-42ee472fe337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.346112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8638b1-9dde-4d15-82c4-42ee472fe337 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.346237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a6ce9-bd66-4453-a4e5-1d7ce03bdc3f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.351041 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aef974-5a7a-449b-94e3-a0b3044524c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.352018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aef974-5a7a-449b-94e3-a0b3044524c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.352608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aef974-5a7a-449b-94e3-a0b3044524c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.352608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aef974-5a7a-449b-94e3-a0b3044524c5 admin admin] PATH_INFO: `/v3/projects/9b5ee0d2e59647dc98721de1b8dfb4db/users/18cd966ad655466e85930f7c505fb360/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.361446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 800/1596] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:17.361446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8638b1-9dde-4d15-82c4-42ee472fe337 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.370730 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aef974-5a7a-449b-94e3-a0b3044524c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5ee0d2e59647dc98721de1b8dfb4db, user_id=18cd966ad655466e85930f7c505fb360, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.371898 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8638b1-9dde-4d15-82c4-42ee472fe337 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.373612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aef974-5a7a-449b-94e3-a0b3044524c5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.378172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bceb4a4-6ea0-4a8a-a910-508fa8e007aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.378966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bceb4a4-6ea0-4a8a-a910-508fa8e007aa admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.379334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bceb4a4-6ea0-4a8a-a910-508fa8e007aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.379655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bceb4a4-6ea0-4a8a-a910-508fa8e007aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.380392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bceb4a4-6ea0-4a8a-a910-508fa8e007aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.382035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bceb4a4-6ea0-4a8a-a910-508fa8e007aa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.388502 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 797/1597] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:17 2022] PUT /identity/v3/projects/9b5ee0d2e59647dc98721de1b8dfb4db/users/18cd966ad655466e85930f7c505fb360/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:17.389338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aef974-5a7a-449b-94e3-a0b3044524c5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.401833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 801/1598] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:17.401833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8592755-f9ea-4e24-80e5-044f0e6d7b07 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.401833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8592755-f9ea-4e24-80e5-044f0e6d7b07 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.401833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8592755-f9ea-4e24-80e5-044f0e6d7b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.421442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:17.430928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e8592755-f9ea-4e24-80e5-044f0e6d7b07 None None] MFA Rules not processed for user `f858d1d451e44a0fbd3fdbff44220c0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:17.432731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e8592755-f9ea-4e24-80e5-044f0e6d7b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:17.485321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aef974-5a7a-449b-94e3-a0b3044524c5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.500351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb386f4-ec9f-4c36-97be-70482d062f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.501883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb386f4-ec9f-4c36-97be-70482d062f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.502288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb386f4-ec9f-4c36-97be-70482d062f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.502869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb386f4-ec9f-4c36-97be-70482d062f1b admin admin] PATH_INFO: `/v3/projects/e916473814d44ac1ae6339f0d052daa1/users/9178e89a790d48d19b25c4c22bd0ca4e/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.522721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb386f4-ec9f-4c36-97be-70482d062f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e916473814d44ac1ae6339f0d052daa1, user_id=9178e89a790d48d19b25c4c22bd0ca4e, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.526780 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb386f4-ec9f-4c36-97be-70482d062f1b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.527187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 802/1599] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:17.528385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8592755-f9ea-4e24-80e5-044f0e6d7b07 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.548159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8592755-f9ea-4e24-80e5-044f0e6d7b07 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.564251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e3aac-32da-407d-874d-90ed7b35cd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.565178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e3aac-32da-407d-874d-90ed7b35cd64 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.565526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e3aac-32da-407d-874d-90ed7b35cd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.565949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e3aac-32da-407d-874d-90ed7b35cd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.566848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e3aac-32da-407d-874d-90ed7b35cd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.567077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 798/1600] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:17 2022] PUT /identity/v3/projects/e916473814d44ac1ae6339f0d052daa1/users/9178e89a790d48d19b25c4c22bd0ca4e/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:17.568794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb386f4-ec9f-4c36-97be-70482d062f1b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.569493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e3aac-32da-407d-874d-90ed7b35cd64 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.587648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 803/1601] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:17.588682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e3aac-32da-407d-874d-90ed7b35cd64 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.601221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e3aac-32da-407d-874d-90ed7b35cd64 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.668571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb386f4-ec9f-4c36-97be-70482d062f1b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.672821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69277c-4d7f-4d56-b679-feb96dc33cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.673976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69277c-4d7f-4d56-b679-feb96dc33cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.674300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd361f-ce38-4358-8805-a011132c4580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.674458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69277c-4d7f-4d56-b679-feb96dc33cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.674825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69277c-4d7f-4d56-b679-feb96dc33cd5 admin admin] PATH_INFO: `/v3/projects/3e8c39550e4e41e79cc56aa7d22ae052/users/15fae427f22d476fa0e378949de41292/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.675684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd361f-ce38-4358-8805-a011132c4580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.676126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd361f-ce38-4358-8805-a011132c4580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.676553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd361f-ce38-4358-8805-a011132c4580 admin admin] PATH_INFO: `/v3/projects/7ae7d1ac52ec41c7a14e107347da454f/users/c773fb31a56f48d196e073f9e5e350a1/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.694277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69277c-4d7f-4d56-b679-feb96dc33cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e8c39550e4e41e79cc56aa7d22ae052, user_id=15fae427f22d476fa0e378949de41292, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.698023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd361f-ce38-4358-8805-a011132c4580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ae7d1ac52ec41c7a14e107347da454f, user_id=c773fb31a56f48d196e073f9e5e350a1, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.698482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69277c-4d7f-4d56-b679-feb96dc33cd5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.701545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd361f-ce38-4358-8805-a011132c4580 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.717091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 804/1602] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:17 2022] PUT /identity/v3/projects/3e8c39550e4e41e79cc56aa7d22ae052/users/15fae427f22d476fa0e378949de41292/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:17.718704 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d84ed-94af-4b48-a0b1-51b79646ff86 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.719134 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d84ed-94af-4b48-a0b1-51b79646ff86 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.719540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d84ed-94af-4b48-a0b1-51b79646ff86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.723787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 799/1603] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:17 2022] PUT /identity/v3/projects/7ae7d1ac52ec41c7a14e107347da454f/users/c773fb31a56f48d196e073f9e5e350a1/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:17.724925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd361f-ce38-4358-8805-a011132c4580 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.739348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-987d84ed-94af-4b48-a0b1-51b79646ff86 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:17.740981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-987d84ed-94af-4b48-a0b1-51b79646ff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:17.779010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:17.824973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd361f-ce38-4358-8805-a011132c4580 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.830899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 805/1604] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:17.832083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d84ed-94af-4b48-a0b1-51b79646ff86 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.840949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e834804-aa26-4dbf-825e-499c8fdf2ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.841754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834804-aa26-4dbf-825e-499c8fdf2ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.842118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834804-aa26-4dbf-825e-499c8fdf2ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.842489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834804-aa26-4dbf-825e-499c8fdf2ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.843298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e834804-aa26-4dbf-825e-499c8fdf2ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.845830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d84ed-94af-4b48-a0b1-51b79646ff86 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.846154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e834804-aa26-4dbf-825e-499c8fdf2ce6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.851784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21550ba2-0488-453d-8436-80eed2302510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:17.852531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21550ba2-0488-453d-8436-80eed2302510 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.852884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21550ba2-0488-453d-8436-80eed2302510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.853229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21550ba2-0488-453d-8436-80eed2302510 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.855646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21550ba2-0488-453d-8436-80eed2302510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:17.858079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21550ba2-0488-453d-8436-80eed2302510 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:17.868127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 800/1605] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:17.868869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31731b8-1e06-4d9d-9f78-d394326405b0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.868869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31731b8-1e06-4d9d-9f78-d394326405b0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.869162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31731b8-1e06-4d9d-9f78-d394326405b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.890618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b31731b8-1e06-4d9d-9f78-d394326405b0 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:17.892349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b31731b8-1e06-4d9d-9f78-d394326405b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:17.902154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 801/1606] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:17.903077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31731b8-1e06-4d9d-9f78-d394326405b0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:17.915064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 806/1607] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:17 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:17.915064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3a1c0-b1a5-4ca6-a2f0-a9cfa2297cd7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:17.915064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3a1c0-b1a5-4ca6-a2f0-a9cfa2297cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:17.915064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3a1c0-b1a5-4ca6-a2f0-a9cfa2297cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:17.924350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31731b8-1e06-4d9d-9f78-d394326405b0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:17.932659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7fd3a1c0-b1a5-4ca6-a2f0-a9cfa2297cd7 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:17.933907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7fd3a1c0-b1a5-4ca6-a2f0-a9cfa2297cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:17.942900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:18.010460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca9a16-8665-4129-96b5-3598fec33c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:18.011550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9a16-8665-4129-96b5-3598fec33c93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.011959 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9a16-8665-4129-96b5-3598fec33c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.012084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 807/1608] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:18.012283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9a16-8665-4129-96b5-3598fec33c93 admin admin] PATH_INFO: `/v3/projects/95d1e25b20a146c198dfeda0dc43cd72/users/30de794b71b343289491531dfbe37f53/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.013705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e984-b45d-4b87-a2e3-0f156e777396 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.014089 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e984-b45d-4b87-a2e3-0f156e777396 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.014447 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e984-b45d-4b87-a2e3-0f156e777396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.036486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca9a16-8665-4129-96b5-3598fec33c93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d1e25b20a146c198dfeda0dc43cd72, user_id=30de794b71b343289491531dfbe37f53, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:18.040610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca9a16-8665-4129-96b5-3598fec33c93 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:18.057114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-49e5e984-b45d-4b87-a2e3-0f156e777396 None None] MFA Rules not processed for user `c773fb31a56f48d196e073f9e5e350a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:18.058319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-49e5e984-b45d-4b87-a2e3-0f156e777396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:18.059882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 802/1609] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:17 2022] PUT /identity/v3/projects/95d1e25b20a146c198dfeda0dc43cd72/users/30de794b71b343289491531dfbe37f53/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:18.060919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca9a16-8665-4129-96b5-3598fec33c93 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:18.062875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-42ca9a16-8665-4129-96b5-3598fec33c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:18.137090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca9a16-8665-4129-96b5-3598fec33c93 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:18.144518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068607f5-5941-4640-83da-1e08fbc46146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:18.145192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068607f5-5941-4640-83da-1e08fbc46146 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.145538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068607f5-5941-4640-83da-1e08fbc46146 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.146282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068607f5-5941-4640-83da-1e08fbc46146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.158168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 808/1610] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:18.158168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5e984-b45d-4b87-a2e3-0f156e777396 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:18.231359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:18.247816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068607f5-5941-4640-83da-1e08fbc46146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:18.250504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068607f5-5941-4640-83da-1e08fbc46146 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:18.267443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5e984-b45d-4b87-a2e3-0f156e777396 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:18.273489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4adf31-9653-4e20-8e69-49f2189b55c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:18.274592 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 803/1611] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:18.274848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4adf31-9653-4e20-8e69-49f2189b55c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.275237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4adf31-9653-4e20-8e69-49f2189b55c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.276142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4adf31-9653-4e20-8e69-49f2189b55c3 admin admin] PATH_INFO: `/v3/projects/88bf6b4fb2cd42108fa32b92c44a5e33/users/cccc62f6775d4a3a9bc19bbd6a3b8082/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.277043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068607f5-5941-4640-83da-1e08fbc46146 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:18.295243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068607f5-5941-4640-83da-1e08fbc46146 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:18.300172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aae438-c78d-42a4-b25f-9a95a9a93fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:18.300906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aae438-c78d-42a4-b25f-9a95a9a93fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.301244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aae438-c78d-42a4-b25f-9a95a9a93fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.301523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aae438-c78d-42a4-b25f-9a95a9a93fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.303111 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4adf31-9653-4e20-8e69-49f2189b55c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88bf6b4fb2cd42108fa32b92c44a5e33, user_id=cccc62f6775d4a3a9bc19bbd6a3b8082, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:18.311268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4adf31-9653-4e20-8e69-49f2189b55c3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:18.317293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aae438-c78d-42a4-b25f-9a95a9a93fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:18.318756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aae438-c78d-42a4-b25f-9a95a9a93fc0 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:18.330852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 804/1612] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:18.331740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aae438-c78d-42a4-b25f-9a95a9a93fc0 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:18.339618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 809/1613] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:18 2022] PUT /identity/v3/projects/88bf6b4fb2cd42108fa32b92c44a5e33/users/cccc62f6775d4a3a9bc19bbd6a3b8082/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:18.342198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4adf31-9653-4e20-8e69-49f2189b55c3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:18.344674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aae438-c78d-42a4-b25f-9a95a9a93fc0 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:18.436842 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:18.481064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4adf31-9653-4e20-8e69-49f2189b55c3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:18.484410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b8d88-3fcd-422f-915f-cd493b29476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:18.487461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b8d88-3fcd-422f-915f-cd493b29476d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.487461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b8d88-3fcd-422f-915f-cd493b29476d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.487461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b8d88-3fcd-422f-915f-cd493b29476d admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.487461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b8d88-3fcd-422f-915f-cd493b29476d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:18.490506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960356b-243b-4c46-8155-85ed06da4160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:18.491172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960356b-243b-4c46-8155-85ed06da4160 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.491459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960356b-243b-4c46-8155-85ed06da4160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.491934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960356b-243b-4c46-8155-85ed06da4160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.493671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b8d88-3fcd-422f-915f-cd493b29476d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:18.526427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960356b-243b-4c46-8155-85ed06da4160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:18.529652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960356b-243b-4c46-8155-85ed06da4160 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:18.533670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 805/1614] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:18 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:18.533670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a743d84-2565-442e-a3aa-5b8954da980e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.534010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a743d84-2565-442e-a3aa-5b8954da980e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.534242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a743d84-2565-442e-a3aa-5b8954da980e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.554706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 810/1615] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:18.558618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f584a7-35d4-4e8b-9a0c-e7622633c932 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.558618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f584a7-35d4-4e8b-9a0c-e7622633c932 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.559033 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f584a7-35d4-4e8b-9a0c-e7622633c932 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.561366 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 811/1616] 10.222.0.42 () {54 vars in 961 bytes} [Wed May 25 16:20:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:20:18.568459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f176-6912-4381-8a3d-f078e830a870 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.568903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f176-6912-4381-8a3d-f078e830a870 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.569632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f176-6912-4381-8a3d-f078e830a870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.578929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7a743d84-2565-442e-a3aa-5b8954da980e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:18.580286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7a743d84-2565-442e-a3aa-5b8954da980e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:18.595912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 806/1617] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:18.596950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a743d84-2565-442e-a3aa-5b8954da980e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:18.608297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a743d84-2565-442e-a3aa-5b8954da980e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:18.614160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b6b76-9aef-41cb-84c9-77404dde54de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:18.614870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b6b76-9aef-41cb-84c9-77404dde54de admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.615534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b6b76-9aef-41cb-84c9-77404dde54de admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.615834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b6b76-9aef-41cb-84c9-77404dde54de admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.616579 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b6b76-9aef-41cb-84c9-77404dde54de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:18.618576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b6b76-9aef-41cb-84c9-77404dde54de admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:18.627402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4d48f176-6912-4381-8a3d-f078e830a870 None None] MFA Rules not processed for user `cccc62f6775d4a3a9bc19bbd6a3b8082`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:18.628988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4d48f176-6912-4381-8a3d-f078e830a870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:18.631165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 807/1618] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:18.632333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df9d87-1829-4666-bd93-f8351f01d921 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.633378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df9d87-1829-4666-bd93-f8351f01d921 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.633671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df9d87-1829-4666-bd93-f8351f01d921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.662030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-77df9d87-1829-4666-bd93-f8351f01d921 None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:18.663626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-77df9d87-1829-4666-bd93-f8351f01d921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:18.725077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 812/1619] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:18.726142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48f176-6912-4381-8a3d-f078e830a870 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:18.752112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48f176-6912-4381-8a3d-f078e830a870 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:18.757115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0eea83-ca41-4248-a8ea-6fd5dff390f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:18.758003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eea83-ca41-4248-a8ea-6fd5dff390f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.758244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eea83-ca41-4248-a8ea-6fd5dff390f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.758563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eea83-ca41-4248-a8ea-6fd5dff390f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.765340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 808/1620] 10.222.0.42 () {58 vars in 1058 bytes} [Wed May 25 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:18.766228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df9d87-1829-4666-bd93-f8351f01d921 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:18.785488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df9d87-1829-4666-bd93-f8351f01d921 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:18.792005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53871118-220c-4a0b-a8e8-8fe0d7518477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:18.794361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53871118-220c-4a0b-a8e8-8fe0d7518477 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.794361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53871118-220c-4a0b-a8e8-8fe0d7518477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.794361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53871118-220c-4a0b-a8e8-8fe0d7518477 admin admin] PATH_INFO: `/v3/projects/37de430b0d774c1799e70a67cbf07db4/users/83c8dd0277cb452ca56e0d5a50ed0cbd/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.808913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0eea83-ca41-4248-a8ea-6fd5dff390f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:18.811080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0eea83-ca41-4248-a8ea-6fd5dff390f1 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:18.823319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53871118-220c-4a0b-a8e8-8fe0d7518477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37de430b0d774c1799e70a67cbf07db4, user_id=83c8dd0277cb452ca56e0d5a50ed0cbd, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:18.824900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 813/1621] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:18.826451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0eea83-ca41-4248-a8ea-6fd5dff390f1 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:18.827653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53871118-220c-4a0b-a8e8-8fe0d7518477 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:18.845772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0eea83-ca41-4248-a8ea-6fd5dff390f1 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:18.861619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 809/1622] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:18 2022] PUT /identity/v3/projects/37de430b0d774c1799e70a67cbf07db4/users/83c8dd0277cb452ca56e0d5a50ed0cbd/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:18.862669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53871118-220c-4a0b-a8e8-8fe0d7518477 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:18.912005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:18.936674 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65adcaa-1fb1-452b-8f6f-70f57390741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:18.936674 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65adcaa-1fb1-452b-8f6f-70f57390741a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:18.937031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65adcaa-1fb1-452b-8f6f-70f57390741a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:18.937293 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65adcaa-1fb1-452b-8f6f-70f57390741a admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:18.938138 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65adcaa-1fb1-452b-8f6f-70f57390741a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:18.940414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65adcaa-1fb1-452b-8f6f-70f57390741a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:18.971757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 814/1623] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:18 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:18.972884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65adcaa-1fb1-452b-8f6f-70f57390741a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:18.977289 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53871118-220c-4a0b-a8e8-8fe0d7518477 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:18.999610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:19.077639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65adcaa-1fb1-452b-8f6f-70f57390741a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:19.082589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b96c49-c539-4253-acd9-7e5a78b10270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:19.083451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b96c49-c539-4253-acd9-7e5a78b10270 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.083809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b96c49-c539-4253-acd9-7e5a78b10270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.084190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b96c49-c539-4253-acd9-7e5a78b10270 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:19.085028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b96c49-c539-4253-acd9-7e5a78b10270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:19.086996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b96c49-c539-4253-acd9-7e5a78b10270 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:19.090588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd81fc-0e39-45cc-8275-767103a5fd7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:19.091514 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd81fc-0e39-45cc-8275-767103a5fd7f service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.091963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd81fc-0e39-45cc-8275-767103a5fd7f service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.092386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd81fc-0e39-45cc-8275-767103a5fd7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:19.101784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 815/1624] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:19.101784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b96c49-c539-4253-acd9-7e5a78b10270 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:19.134814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b96c49-c539-4253-acd9-7e5a78b10270 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:19.146390 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326768e5-4295-4671-a9e0-b683166f4483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:19.148282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326768e5-4295-4671-a9e0-b683166f4483 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.148828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326768e5-4295-4671-a9e0-b683166f4483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.149294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326768e5-4295-4671-a9e0-b683166f4483 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:19.151926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326768e5-4295-4671-a9e0-b683166f4483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:19.156105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326768e5-4295-4671-a9e0-b683166f4483 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:19.177164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd81fc-0e39-45cc-8275-767103a5fd7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:19.179228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd81fc-0e39-45cc-8275-767103a5fd7f service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:19.195267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 816/1625] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:19 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:19.197310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326768e5-4295-4671-a9e0-b683166f4483 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:19.205698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 810/1626] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:19.207045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd81fc-0e39-45cc-8275-767103a5fd7f service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:19.219055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326768e5-4295-4671-a9e0-b683166f4483 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:19.228319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd81fc-0e39-45cc-8275-767103a5fd7f service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:19.234415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e5066-4507-4c54-8c5e-932635d948d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:19.235206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e5066-4507-4c54-8c5e-932635d948d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.235593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e5066-4507-4c54-8c5e-932635d948d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.235970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e5066-4507-4c54-8c5e-932635d948d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:19.236208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479973b-67c7-4658-9420-4298b9d90e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:19.237203 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e5066-4507-4c54-8c5e-932635d948d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:19.238247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479973b-67c7-4658-9420-4298b9d90e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.239374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479973b-67c7-4658-9420-4298b9d90e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.241601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479973b-67c7-4658-9420-4298b9d90e6d admin admin] PATH_INFO: `/v3/projects/37de430b0d774c1799e70a67cbf07db4/users/83c8dd0277cb452ca56e0d5a50ed0cbd/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:19.243940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e5066-4507-4c54-8c5e-932635d948d6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:19.247844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479973b-67c7-4658-9420-4298b9d90e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=37de430b0d774c1799e70a67cbf07db4, user_id=83c8dd0277cb452ca56e0d5a50ed0cbd, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:19.254044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479973b-67c7-4658-9420-4298b9d90e6d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:19.257019 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 811/1627] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:19.273962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e5066-4507-4c54-8c5e-932635d948d6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:19.277600 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 817/1628] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:19 2022] PUT /identity/v3/projects/37de430b0d774c1799e70a67cbf07db4/users/83c8dd0277cb452ca56e0d5a50ed0cbd/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:19.289324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8846298-a853-40d3-8786-1220425c4d7c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.290000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8846298-a853-40d3-8786-1220425c4d7c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.290518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8846298-a853-40d3-8786-1220425c4d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:19.297912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e5066-4507-4c54-8c5e-932635d948d6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:19.371580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e8846298-a853-40d3-8786-1220425c4d7c None None] MFA Rules not processed for user `83c8dd0277cb452ca56e0d5a50ed0cbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:19.377513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e8846298-a853-40d3-8786-1220425c4d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:19.384289 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24ae48-7eee-4bb0-9a77-e378252521f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:19.386290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24ae48-7eee-4bb0-9a77-e378252521f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.386290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24ae48-7eee-4bb0-9a77-e378252521f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.387003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24ae48-7eee-4bb0-9a77-e378252521f0 admin admin] PATH_INFO: `/v3/projects/afa05e9e7bb3416c9c82df26e29b711b/users/d0621dce94bc43b0956f22c68ce08db6/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:19.410937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24ae48-7eee-4bb0-9a77-e378252521f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa05e9e7bb3416c9c82df26e29b711b, user_id=d0621dce94bc43b0956f22c68ce08db6, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:19.414138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24ae48-7eee-4bb0-9a77-e378252521f0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:19.438757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 812/1629] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:19 2022] PUT /identity/v3/projects/afa05e9e7bb3416c9c82df26e29b711b/users/d0621dce94bc43b0956f22c68ce08db6/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:20:19.443113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24ae48-7eee-4bb0-9a77-e378252521f0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:19.523268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:19.631885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24ae48-7eee-4bb0-9a77-e378252521f0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:19.637858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4be4f-1b95-4569-bf1b-3538b3a92b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:19.638843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4be4f-1b95-4569-bf1b-3538b3a92b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.639360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4be4f-1b95-4569-bf1b-3538b3a92b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.639986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4be4f-1b95-4569-bf1b-3538b3a92b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:19.640939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4be4f-1b95-4569-bf1b-3538b3a92b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:19.642837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4be4f-1b95-4569-bf1b-3538b3a92b8a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:19.690671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 813/1630] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:19.692172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4be4f-1b95-4569-bf1b-3538b3a92b8a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:19.695529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 818/1631] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:20:19.712186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8846298-a853-40d3-8786-1220425c4d7c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:19.722229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4be4f-1b95-4569-bf1b-3538b3a92b8a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:19.745859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8368c-bb6d-494c-aea2-7ab14f3970e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:19.746886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8368c-bb6d-494c-aea2-7ab14f3970e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.747288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8368c-bb6d-494c-aea2-7ab14f3970e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.747668 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8368c-bb6d-494c-aea2-7ab14f3970e7 admin admin] PATH_INFO: `/v3/projects/afa05e9e7bb3416c9c82df26e29b711b/users/d0621dce94bc43b0956f22c68ce08db6/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:19.754615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8368c-bb6d-494c-aea2-7ab14f3970e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa05e9e7bb3416c9c82df26e29b711b, user_id=d0621dce94bc43b0956f22c68ce08db6, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:19.766421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8368c-bb6d-494c-aea2-7ab14f3970e7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:19.772586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:19.806228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 814/1632] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:19 2022] PUT /identity/v3/projects/afa05e9e7bb3416c9c82df26e29b711b/users/d0621dce94bc43b0956f22c68ce08db6/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:19.808687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e0f2d-b98e-4fc2-bffb-82cb8dd8b3e7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.813692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e0f2d-b98e-4fc2-bffb-82cb8dd8b3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.818349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e0f2d-b98e-4fc2-bffb-82cb8dd8b3e7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:19.820160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 815/1633] 10.222.0.42 () {54 vars in 912 bytes} [Wed May 25 16:20:19 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:20:19.821365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c80aa-bd00-4a11-94c5-0e1f0a5434a9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.821689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c80aa-bd00-4a11-94c5-0e1f0a5434a9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.826444 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c80aa-bd00-4a11-94c5-0e1f0a5434a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:19.849302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8846298-a853-40d3-8786-1220425c4d7c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:19.892920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8f7c80aa-bd00-4a11-94c5-0e1f0a5434a9 None None] MFA Rules not processed for user `d0621dce94bc43b0956f22c68ce08db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:19.894227 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8f7c80aa-bd00-4a11-94c5-0e1f0a5434a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:19.916804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f679ad-71d9-4c2b-a114-11f43c2a7948 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:19.917910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f679ad-71d9-4c2b-a114-11f43c2a7948 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:19.917910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f679ad-71d9-4c2b-a114-11f43c2a7948 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:19.918202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f679ad-71d9-4c2b-a114-11f43c2a7948 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:20.031434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f679ad-71d9-4c2b-a114-11f43c2a7948 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:20.033174 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f679ad-71d9-4c2b-a114-11f43c2a7948 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:20.048541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 819/1634] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:20.050005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46585e4-1a80-41fb-b06a-a2e5ae337a36 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:20.053900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46585e4-1a80-41fb-b06a-a2e5ae337a36 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:20.054341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46585e4-1a80-41fb-b06a-a2e5ae337a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:20.061367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 816/1635] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:19 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:20.083991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c80aa-bd00-4a11-94c5-0e1f0a5434a9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:20.098114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c80aa-bd00-4a11-94c5-0e1f0a5434a9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:20.106856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e46585e4-1a80-41fb-b06a-a2e5ae337a36 None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:20.109591 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01681e0-5f4e-44fa-ac23-1e0dfe3fb155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:20.111000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e46585e4-1a80-41fb-b06a-a2e5ae337a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:20.112513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01681e0-5f4e-44fa-ac23-1e0dfe3fb155 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:20.114420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01681e0-5f4e-44fa-ac23-1e0dfe3fb155 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:20.114706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01681e0-5f4e-44fa-ac23-1e0dfe3fb155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:20.139971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01681e0-5f4e-44fa-ac23-1e0dfe3fb155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:20.143814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01681e0-5f4e-44fa-ac23-1e0dfe3fb155 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:20.192502 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 817/1636] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:20 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:20.254403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01681e0-5f4e-44fa-ac23-1e0dfe3fb155 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:20.279760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01681e0-5f4e-44fa-ac23-1e0dfe3fb155 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:20.288143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29207041-0b0d-4faa-a955-4b836f06c8ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:20.290677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 820/1637] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:20:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:20.303764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29207041-0b0d-4faa-a955-4b836f06c8ee service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:20.304368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29207041-0b0d-4faa-a955-4b836f06c8ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:20.304667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29207041-0b0d-4faa-a955-4b836f06c8ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:20.342979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46585e4-1a80-41fb-b06a-a2e5ae337a36 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:20.356978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46585e4-1a80-41fb-b06a-a2e5ae337a36 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:20.367194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b449b-a00b-4818-aacf-c3ca6a763bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:20.367935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b449b-a00b-4818-aacf-c3ca6a763bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:20.368223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b449b-a00b-4818-aacf-c3ca6a763bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:20.368499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b449b-a00b-4818-aacf-c3ca6a763bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:20.395408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b449b-a00b-4818-aacf-c3ca6a763bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:20.397935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b449b-a00b-4818-aacf-c3ca6a763bb7 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:20.428764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 821/1638] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:20.472782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29207041-0b0d-4faa-a955-4b836f06c8ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:20.475275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29207041-0b0d-4faa-a955-4b836f06c8ee service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:20.506046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 818/1639] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:20:21.613115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173b3fc-2125-48e1-9b20-de5e1fc7dd3a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:21.613115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173b3fc-2125-48e1-9b20-de5e1fc7dd3a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:21.613115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173b3fc-2125-48e1-9b20-de5e1fc7dd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:21.620282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:21.676155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b173b3fc-2125-48e1-9b20-de5e1fc7dd3a None None] MFA Rules not processed for user `18cd966ad655466e85930f7c505fb360`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:21.677405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b173b3fc-2125-48e1-9b20-de5e1fc7dd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:21.834288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29207041-0b0d-4faa-a955-4b836f06c8ee service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:21.861977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:21.876059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 822/1640] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:21.888833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b86b3-0c53-4e51-ba53-00502547b2d1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:21.889079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b86b3-0c53-4e51-ba53-00502547b2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:21.889189 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b86b3-0c53-4e51-ba53-00502547b2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:21.924508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-632b86b3-0c53-4e51-ba53-00502547b2d1 None None] MFA Rules not processed for user `18cd966ad655466e85930f7c505fb360`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:21.927890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-632b86b3-0c53-4e51-ba53-00502547b2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:21.950141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 823/1641] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:20:21.965309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29207041-0b0d-4faa-a955-4b836f06c8ee service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:21.970701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:21.972504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:21.973007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:21.973607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] PATH_INFO: `/v3/users/cccc62f6775d4a3a9bc19bbd6a3b8082` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:21.975350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cccc62f6775d4a3a9bc19bbd6a3b8082)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:21.977614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:20:21.977614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 25 16:20:21.977614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: May 25 16:20:21.979072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:21.997739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d722583-9f3c-4a56-aefe-9c01b545fff0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:21.998031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d722583-9f3c-4a56-aefe-9c01b545fff0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:21.998323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d722583-9f3c-4a56-aefe-9c01b545fff0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:22.000225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 824/1642] 10.222.0.42 () {54 vars in 961 bytes} [Wed May 25 16:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:20:22.023660 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b89e4-edaa-4b38-ac73-94a19296a415 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:22.023857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b89e4-edaa-4b38-ac73-94a19296a415 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:22.024052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b89e4-edaa-4b38-ac73-94a19296a415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:22.069744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b81b89e4-edaa-4b38-ac73-94a19296a415 None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:22.071860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b81b89e4-edaa-4b38-ac73-94a19296a415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:22.126042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 825/1643] 10.222.0.42 () {58 vars in 1058 bytes} [Wed May 25 16:20:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:20:22.134675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b89e4-edaa-4b38-ac73-94a19296a415 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:22.139813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cccc62f6775d4a3a9bc19bbd6a3b8082'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:22.159725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b89e4-edaa-4b38-ac73-94a19296a415 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:22.160694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cccc62f6775d4a3a9bc19bbd6a3b8082'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:22.161453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cccc62f6775d4a3a9bc19bbd6a3b8082'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:22.187905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf026c-bbd9-46e1-aa13-9e306da2742b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:22.189926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf026c-bbd9-46e1-aa13-9e306da2742b service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:22.190074 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cccc62f6775d4a3a9bc19bbd6a3b8082'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:22.190420 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf026c-bbd9-46e1-aa13-9e306da2742b service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:22.190822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf026c-bbd9-46e1-aa13-9e306da2742b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:22.193197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a0cf026c-bbd9-46e1-aa13-9e306da2742b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:22.203580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ea8726e5-1331-4fce-9938-5817132fbc5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cccc62f6775d4a3a9bc19bbd6a3b8082'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:22.216473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf026c-bbd9-46e1-aa13-9e306da2742b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:22.217590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 819/1644] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:20:21 2022] DELETE /identity/v3/users/cccc62f6775d4a3a9bc19bbd6a3b8082 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:22.218480 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf026c-bbd9-46e1-aa13-9e306da2742b service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:22.222251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f5cc6-0ec6-49dd-abc7-fc7b9f54b629 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:22.223634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f5cc6-0ec6-49dd-abc7-fc7b9f54b629 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:22.223978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f5cc6-0ec6-49dd-abc7-fc7b9f54b629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:22.236368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 826/1645] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:20:22.252846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-468f5cc6-0ec6-49dd-abc7-fc7b9f54b629 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:22.254445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-468f5cc6-0ec6-49dd-abc7-fc7b9f54b629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:22.340941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 820/1646] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:22.570526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf026c-bbd9-46e1-aa13-9e306da2742b service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:22.572090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a0cf026c-bbd9-46e1-aa13-9e306da2742b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:22.752775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf026c-bbd9-46e1-aa13-9e306da2742b service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:22.891942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14359f0a-308d-455b-b43d-3ebfa460baa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:22.892619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14359f0a-308d-455b-b43d-3ebfa460baa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:22.892997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14359f0a-308d-455b-b43d-3ebfa460baa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:22.893201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14359f0a-308d-455b-b43d-3ebfa460baa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:22.919597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14359f0a-308d-455b-b43d-3ebfa460baa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:22.929182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14359f0a-308d-455b-b43d-3ebfa460baa9 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:22.982528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05104115-6bea-4683-95ee-cae06527443d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:22.982528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05104115-6bea-4683-95ee-cae06527443d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:22.982528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05104115-6bea-4683-95ee-cae06527443d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:22.986135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 827/1647] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:22.986283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 821/1648] 10.222.0.42 () {54 vars in 912 bytes} [Wed May 25 16:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 16:20:22.986460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17438229-a32d-4573-a9d8-4dd78fbb3c0d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:22.986460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17438229-a32d-4573-a9d8-4dd78fbb3c0d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:22.986460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17438229-a32d-4573-a9d8-4dd78fbb3c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:23.019229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-17438229-a32d-4573-a9d8-4dd78fbb3c0d None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:23.020479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-17438229-a32d-4573-a9d8-4dd78fbb3c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:23.052632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:23.147234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 828/1649] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:20:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:23.164171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05104115-6bea-4683-95ee-cae06527443d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:23.169831 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-05104115-6bea-4683-95ee-cae06527443d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:23.301962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:23.362357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05104115-6bea-4683-95ee-cae06527443d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:23.371345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b5972-face-4871-b0c8-3ad166b23441 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8386ca93fe48460ba5b6f0c10dc51122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:23.372034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b5972-face-4871-b0c8-3ad166b23441 service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:23.376368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b5972-face-4871-b0c8-3ad166b23441 service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:23.376368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b5972-face-4871-b0c8-3ad166b23441 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:23.413306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b5972-face-4871-b0c8-3ad166b23441 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:23.415319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b5972-face-4871-b0c8-3ad166b23441 service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:23.442897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 822/1650] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:24.711676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06d9d0-deff-478b-9fbe-c7e59c99c118 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:24.712172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06d9d0-deff-478b-9fbe-c7e59c99c118 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:24.712243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06d9d0-deff-478b-9fbe-c7e59c99c118 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:24.713880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 829/1651] 10.222.0.42 () {54 vars in 912 bytes} [Wed May 25 16:20:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:20:24.727511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f2afe-2a52-4534-824d-8dc241c2a939 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:24.728058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f2afe-2a52-4534-824d-8dc241c2a939 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:24.728217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f2afe-2a52-4534-824d-8dc241c2a939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:24.801085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b74f2afe-2a52-4534-824d-8dc241c2a939 None None] MFA Rules not processed for user `e42c796dbd7441289a7cbc328abeaf81`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:24.803577 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b74f2afe-2a52-4534-824d-8dc241c2a939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:24.826965 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 823/1652] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:20:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:24.944065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06d9d0-deff-478b-9fbe-c7e59c99c118 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:24.962307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06d9d0-deff-478b-9fbe-c7e59c99c118 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:24.972381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e35bef-83d3-45bc-9248-83609fce0b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:24.973056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e35bef-83d3-45bc-9248-83609fce0b22 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:24.973343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e35bef-83d3-45bc-9248-83609fce0b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:24.973620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e35bef-83d3-45bc-9248-83609fce0b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:24.992891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e35bef-83d3-45bc-9248-83609fce0b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:24.999462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e35bef-83d3-45bc-9248-83609fce0b22 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:25.024156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 830/1653] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:25.153515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f2afe-2a52-4534-824d-8dc241c2a939 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:25.155136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b74f2afe-2a52-4534-824d-8dc241c2a939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:25.185633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f2afe-2a52-4534-824d-8dc241c2a939 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:25.194128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:25.200750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:25.201117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:25.201365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] PATH_INFO: `/v3/projects/88bf6b4fb2cd42108fa32b92c44a5e33` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:25.206971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=88bf6b4fb2cd42108fa32b92c44a5e33)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:25.208982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:25.299585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e35bef-83d3-45bc-9248-83609fce0b22 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:25.301149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a1e35bef-83d3-45bc-9248-83609fce0b22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:25.337014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e35bef-83d3-45bc-9248-83609fce0b22 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:25.362267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88bf6b4fb2cd42108fa32b92c44a5e33', 'request_id': 'req-c6ca1a33-97c2-420c-b873-d796516e01ff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.377703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88bf6b4fb2cd42108fa32b92c44a5e33', 'request_id': 'req-c6ca1a33-97c2-420c-b873-d796516e01ff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.388150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88bf6b4fb2cd42108fa32b92c44a5e33', 'request_id': 'req-c6ca1a33-97c2-420c-b873-d796516e01ff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.421696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] The token cache is being invalidate because project 88bf6b4fb2cd42108fa32b92c44a5e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:20:25.424228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.435212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.442794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 824/1654] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:20:25 2022] DELETE /identity/v3/projects/88bf6b4fb2cd42108fa32b92c44a5e33 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:25.443785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:25.446185 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:25.502269 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fa2e2-1f03-4462-8d8e-c292fbf6fa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:25.503523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fa2e2-1f03-4462-8d8e-c292fbf6fa10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:25.504746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fa2e2-1f03-4462-8d8e-c292fbf6fa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:25.505318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fa2e2-1f03-4462-8d8e-c292fbf6fa10 admin admin] PATH_INFO: `/v3/users/c773fb31a56f48d196e073f9e5e350a1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:25.513540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fa2e2-1f03-4462-8d8e-c292fbf6fa10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c773fb31a56f48d196e073f9e5e350a1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:25.518155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fa2e2-1f03-4462-8d8e-c292fbf6fa10 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:25.569942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca1a33-97c2-420c-b873-d796516e01ff admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:25.577020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:25.578199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:25.578575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:25.579045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] PATH_INFO: `/v3/users/d0621dce94bc43b0956f22c68ce08db6` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:25.581078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0621dce94bc43b0956f22c68ce08db6)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:25.586269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:25.630332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-056fa2e2-1f03-4462-8d8e-c292fbf6fa10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c773fb31a56f48d196e073f9e5e350a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.634529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-056fa2e2-1f03-4462-8d8e-c292fbf6fa10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c773fb31a56f48d196e073f9e5e350a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.636397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-056fa2e2-1f03-4462-8d8e-c292fbf6fa10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c773fb31a56f48d196e073f9e5e350a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.672926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-056fa2e2-1f03-4462-8d8e-c292fbf6fa10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c773fb31a56f48d196e073f9e5e350a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.692096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:25.692373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-056fa2e2-1f03-4462-8d8e-c292fbf6fa10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c773fb31a56f48d196e073f9e5e350a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.696950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0621dce94bc43b0956f22c68ce08db6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.723697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 831/1655] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:20:25 2022] DELETE /identity/v3/users/c773fb31a56f48d196e073f9e5e350a1 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:20:25.727049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0621dce94bc43b0956f22c68ce08db6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.727049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0621dce94bc43b0956f22c68ce08db6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.729454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d01ac-1299-4b43-8ce5-618932cc32fe None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:25.742947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d01ac-1299-4b43-8ce5-618932cc32fe None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:25.743574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d01ac-1299-4b43-8ce5-618932cc32fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:25.821669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0621dce94bc43b0956f22c68ce08db6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.835649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-510d01ac-1299-4b43-8ce5-618932cc32fe None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:25.837279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-510d01ac-1299-4b43-8ce5-618932cc32fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:25.872955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0621dce94bc43b0956f22c68ce08db6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:25.911889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 825/1656] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:20:25 2022] DELETE /identity/v3/users/d0621dce94bc43b0956f22c68ce08db6 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:25.917020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:26.001677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 832/1657] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:26.034378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:26.035894 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:26.168474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3696ec-5f58-4e5b-8e47-174056a74d69 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:26.176503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe90e2-8aa8-4d00-be0b-75fb19c6d49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:26.177424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe90e2-8aa8-4d00-be0b-75fb19c6d49f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:26.178653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe90e2-8aa8-4d00-be0b-75fb19c6d49f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:26.179347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe90e2-8aa8-4d00-be0b-75fb19c6d49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:26.273475 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe90e2-8aa8-4d00-be0b-75fb19c6d49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:26.275253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe90e2-8aa8-4d00-be0b-75fb19c6d49f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:26.292638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 826/1658] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:26.860516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007e13-486a-4f2f-826c-5577ec799699 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:26.861349 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007e13-486a-4f2f-826c-5577ec799699 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:26.861822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007e13-486a-4f2f-826c-5577ec799699 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:26.863880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 833/1659] 10.222.0.42 () {54 vars in 960 bytes} [Wed May 25 16:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:20:26.884362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e347e5-fda2-44ea-8681-b44760388058 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:26.884941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e347e5-fda2-44ea-8681-b44760388058 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:26.885653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e347e5-fda2-44ea-8681-b44760388058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:26.921252 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-15e347e5-fda2-44ea-8681-b44760388058 None None] MFA Rules not processed for user `6719421144734ef49b3809df30a45961`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:26.922544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-15e347e5-fda2-44ea-8681-b44760388058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:27.078163 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 827/1660] 10.222.0.42 () {58 vars in 1057 bytes} [Wed May 25 16:20:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:27.078536 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5007e13-486a-4f2f-826c-5577ec799699 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:27.107376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5007e13-486a-4f2f-826c-5577ec799699 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:27.116608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6dd69-cabc-4251-abec-b8edd629260d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:27.117937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6dd69-cabc-4251-abec-b8edd629260d service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:27.118562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6dd69-cabc-4251-abec-b8edd629260d service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:27.119151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6dd69-cabc-4251-abec-b8edd629260d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:27.123295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b7e6dd69-cabc-4251-abec-b8edd629260d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:27.154630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6dd69-cabc-4251-abec-b8edd629260d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:27.157514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6dd69-cabc-4251-abec-b8edd629260d service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:27.186372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 834/1661] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:20:28.280960 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e347e5-fda2-44ea-8681-b44760388058 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:28.282641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-15e347e5-fda2-44ea-8681-b44760388058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:28.307379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e347e5-fda2-44ea-8681-b44760388058 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:28.319307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:28.321650 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:28.323040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:28.323381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] PATH_INFO: `/v3/projects/afa05e9e7bb3416c9c82df26e29b711b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:28.327634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa05e9e7bb3416c9c82df26e29b711b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:28.331995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:28.343861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:28.467930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa05e9e7bb3416c9c82df26e29b711b', 'request_id': 'req-a2a708ee-6248-4d28-b263-7d67cbe5921b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:28.493670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39361c7-5884-492e-b20c-bbe39b8f032b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:28.494324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39361c7-5884-492e-b20c-bbe39b8f032b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:28.494467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39361c7-5884-492e-b20c-bbe39b8f032b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:28.499268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa05e9e7bb3416c9c82df26e29b711b', 'request_id': 'req-a2a708ee-6248-4d28-b263-7d67cbe5921b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:28.517937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa05e9e7bb3416c9c82df26e29b711b', 'request_id': 'req-a2a708ee-6248-4d28-b263-7d67cbe5921b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:28.560072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] The token cache is being invalidate because project afa05e9e7bb3416c9c82df26e29b711b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:20:28.560753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:28.561678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2a708ee-6248-4d28-b263-7d67cbe5921b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:28.562414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a39361c7-5884-492e-b20c-bbe39b8f032b None None] MFA Rules not processed for user `b781c4c6cb61439fb781988a7da028ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:28.563752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a39361c7-5884-492e-b20c-bbe39b8f032b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:28.565425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 828/1662] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:20:28 2022] DELETE /identity/v3/projects/afa05e9e7bb3416c9c82df26e29b711b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:28.570900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582233c-5739-46a1-ba23-f4412f59a340 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:28.571197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582233c-5739-46a1-ba23-f4412f59a340 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:28.571901 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582233c-5739-46a1-ba23-f4412f59a340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:28.616811 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7582233c-5739-46a1-ba23-f4412f59a340 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:28.618847 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7582233c-5739-46a1-ba23-f4412f59a340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:28.723854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 835/1663] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:28.725052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39361c7-5884-492e-b20c-bbe39b8f032b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:28.726555 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a39361c7-5884-492e-b20c-bbe39b8f032b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:28.766583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 829/1664] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:28.767804 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582233c-5739-46a1-ba23-f4412f59a340 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:28.770207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7582233c-5739-46a1-ba23-f4412f59a340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:28.879582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39361c7-5884-492e-b20c-bbe39b8f032b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:28.886316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:28.887853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:28.888322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:28.888866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] PATH_INFO: `/v3/projects/7ae7d1ac52ec41c7a14e107347da454f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:28.891082 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ae7d1ac52ec41c7a14e107347da454f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:28.892453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:20:28.892453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 25 16:20:28.892453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: May 25 16:20:28.894138 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:28.991864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:29.007368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582233c-5739-46a1-ba23-f4412f59a340 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:29.013029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ae7d1ac52ec41c7a14e107347da454f', 'request_id': 'req-3aeb5b07-5518-40bc-995c-76a567a8fddf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:29.044388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ae7d1ac52ec41c7a14e107347da454f', 'request_id': 'req-3aeb5b07-5518-40bc-995c-76a567a8fddf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:29.061842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ae7d1ac52ec41c7a14e107347da454f', 'request_id': 'req-3aeb5b07-5518-40bc-995c-76a567a8fddf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:29.084399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:29.094230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] The token cache is being invalidate because project 7ae7d1ac52ec41c7a14e107347da454f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:20:29.094530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:29.098471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3aeb5b07-5518-40bc-995c-76a567a8fddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:29.101440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 836/1665] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:20:28 2022] DELETE /identity/v3/projects/7ae7d1ac52ec41c7a14e107347da454f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:29.112295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d1cc5-d18d-4074-84b7-e733d76d252e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:29.112603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d1cc5-d18d-4074-84b7-e733d76d252e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:29.112855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d1cc5-d18d-4074-84b7-e733d76d252e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:29.168394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-681d1cc5-d18d-4074-84b7-e733d76d252e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:29.175132 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-681d1cc5-d18d-4074-84b7-e733d76d252e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:29.199147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2f90d-28b5-468f-be9e-eb7742003669 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:29.199838 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2f90d-28b5-468f-be9e-eb7742003669 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:29.200123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2f90d-28b5-468f-be9e-eb7742003669 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:29.200399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2f90d-28b5-468f-be9e-eb7742003669 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:29.210040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b0c2f90d-28b5-468f-be9e-eb7742003669 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:29.393457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 837/1666] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:29.393457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782069c-6643-428f-a3f9-748885b6ae8a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:29.393457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782069c-6643-428f-a3f9-748885b6ae8a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:29.393457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782069c-6643-428f-a3f9-748885b6ae8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:29.436221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5782069c-6643-428f-a3f9-748885b6ae8a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:29.437917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5782069c-6643-428f-a3f9-748885b6ae8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:29.438881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2f90d-28b5-468f-be9e-eb7742003669 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:29.442181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2f90d-28b5-468f-be9e-eb7742003669 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:29.458695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 838/1667] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:29.467111 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782069c-6643-428f-a3f9-748885b6ae8a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:29.470231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 830/1668] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:29.471979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ab2e0-452a-4bdf-b0be-93f5ffc17475 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:29.472512 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ab2e0-452a-4bdf-b0be-93f5ffc17475 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:29.472985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ab2e0-452a-4bdf-b0be-93f5ffc17475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:29.515014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782069c-6643-428f-a3f9-748885b6ae8a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:29.544647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de2319-d1b8-4c7c-8f9b-54f49fa75b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:29.545344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de2319-d1b8-4c7c-8f9b-54f49fa75b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:29.545871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de2319-d1b8-4c7c-8f9b-54f49fa75b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:29.546162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de2319-d1b8-4c7c-8f9b-54f49fa75b60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:29.546910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de2319-d1b8-4c7c-8f9b-54f49fa75b60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:29.547220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de2319-d1b8-4c7c-8f9b-54f49fa75b60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:29.551951 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de2319-d1b8-4c7c-8f9b-54f49fa75b60 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:29.561776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-673ab2e0-452a-4bdf-b0be-93f5ffc17475 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:29.563311 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-673ab2e0-452a-4bdf-b0be-93f5ffc17475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:29.595765 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 839/1669] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:29.596704 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de2319-d1b8-4c7c-8f9b-54f49fa75b60 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:29.611423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 831/1670] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:29.614932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ab2e0-452a-4bdf-b0be-93f5ffc17475 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:29.629472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de2319-d1b8-4c7c-8f9b-54f49fa75b60 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:29.647454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ab2e0-452a-4bdf-b0be-93f5ffc17475 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:29.659730 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62304423-ed9f-4020-b3a7-5d5d4595ee34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:29.663289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62304423-ed9f-4020-b3a7-5d5d4595ee34 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:29.664223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62304423-ed9f-4020-b3a7-5d5d4595ee34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:29.666293 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62304423-ed9f-4020-b3a7-5d5d4595ee34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:29.669426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62304423-ed9f-4020-b3a7-5d5d4595ee34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:29.679161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a06101-11d5-402b-ad60-7b43c25d5e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:29.680004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06101-11d5-402b-ad60-7b43c25d5e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:29.680245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62304423-ed9f-4020-b3a7-5d5d4595ee34 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:29.680479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06101-11d5-402b-ad60-7b43c25d5e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:29.681032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06101-11d5-402b-ad60-7b43c25d5e57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:29.682534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a06101-11d5-402b-ad60-7b43c25d5e57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:29.684217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a06101-11d5-402b-ad60-7b43c25d5e57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:29.687059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a06101-11d5-402b-ad60-7b43c25d5e57 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:29.729840 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 832/1671] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:29.730904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a06101-11d5-402b-ad60-7b43c25d5e57 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:29.758793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a06101-11d5-402b-ad60-7b43c25d5e57 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:29.783981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4730660-76aa-45ea-8f8d-0966cfb1f6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:29.784679 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730660-76aa-45ea-8f8d-0966cfb1f6aa admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:29.784964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730660-76aa-45ea-8f8d-0966cfb1f6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:29.785242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730660-76aa-45ea-8f8d-0966cfb1f6aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:29.785976 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4730660-76aa-45ea-8f8d-0966cfb1f6aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:29.795398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 840/1672] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:29 2022] POST /identity/v3/projects => generated 387 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:29.796473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62304423-ed9f-4020-b3a7-5d5d4595ee34 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:29.803494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4730660-76aa-45ea-8f8d-0966cfb1f6aa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:29.901408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 833/1673] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:29 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:29.902292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4730660-76aa-45ea-8f8d-0966cfb1f6aa admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:29.996682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62304423-ed9f-4020-b3a7-5d5d4595ee34 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:30.059801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4730660-76aa-45ea-8f8d-0966cfb1f6aa admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:30.070378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a2791-1641-4edd-a96e-ea67f1502dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:30.075152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:30.080991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a2791-1641-4edd-a96e-ea67f1502dde admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:30.081604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a2791-1641-4edd-a96e-ea67f1502dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:30.082093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a2791-1641-4edd-a96e-ea67f1502dde admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:30.083148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a2791-1641-4edd-a96e-ea67f1502dde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:30.088316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a2791-1641-4edd-a96e-ea67f1502dde admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:30.118331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb351a1-cda2-4030-b119-de460279e7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:30.119290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb351a1-cda2-4030-b119-de460279e7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:30.119759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb351a1-cda2-4030-b119-de460279e7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:30.120043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb351a1-cda2-4030-b119-de460279e7e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:30.120900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb351a1-cda2-4030-b119-de460279e7e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:30.123086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb351a1-cda2-4030-b119-de460279e7e3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:30.134216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 834/1674] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:29 2022] POST /identity/v3/users => generated 283 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:30.136327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a2791-1641-4edd-a96e-ea67f1502dde admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:30.164129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 841/1675] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:29 2022] POST /identity/v3/users => generated 294 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:30.165663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb351a1-cda2-4030-b119-de460279e7e3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:30.190650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb351a1-cda2-4030-b119-de460279e7e3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:30.195712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a2791-1641-4edd-a96e-ea67f1502dde admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:30.203109 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282993fd-30f3-42e2-871a-52903571d0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:30.204884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282993fd-30f3-42e2-871a-52903571d0ee admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:30.205868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282993fd-30f3-42e2-871a-52903571d0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:30.206801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282993fd-30f3-42e2-871a-52903571d0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:30.210838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282993fd-30f3-42e2-871a-52903571d0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:30.213329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282993fd-30f3-42e2-871a-52903571d0ee admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:30.217778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b497a47-90d3-49bb-ae6f-338e89519c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:30.217778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b497a47-90d3-49bb-ae6f-338e89519c19 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:30.217778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b497a47-90d3-49bb-ae6f-338e89519c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:30.217778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b497a47-90d3-49bb-ae6f-338e89519c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:30.218149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b497a47-90d3-49bb-ae6f-338e89519c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:30.232504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b497a47-90d3-49bb-ae6f-338e89519c19 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:30.245364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 842/1676] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:30.246461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282993fd-30f3-42e2-871a-52903571d0ee admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:30.259255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 835/1677] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:30.260340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b497a47-90d3-49bb-ae6f-338e89519c19 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:30.266715 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282993fd-30f3-42e2-871a-52903571d0ee admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:30.288706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547da152-9890-41b4-8a3b-bc7a7ef2b5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:30.289475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547da152-9890-41b4-8a3b-bc7a7ef2b5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:30.289933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547da152-9890-41b4-8a3b-bc7a7ef2b5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:30.290294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547da152-9890-41b4-8a3b-bc7a7ef2b5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:30.291792 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547da152-9890-41b4-8a3b-bc7a7ef2b5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:30.293611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547da152-9890-41b4-8a3b-bc7a7ef2b5d0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:30.296843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:30.313011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b497a47-90d3-49bb-ae6f-338e89519c19 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:30.321947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 843/1678] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:30.324996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547da152-9890-41b4-8a3b-bc7a7ef2b5d0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:30.340012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5422c-64f2-4ed0-b950-d193b2cb4759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:30.341673 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5422c-64f2-4ed0-b950-d193b2cb4759 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:30.341995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5422c-64f2-4ed0-b950-d193b2cb4759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:30.342431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5422c-64f2-4ed0-b950-d193b2cb4759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:30.343745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5422c-64f2-4ed0-b950-d193b2cb4759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:30.345391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5422c-64f2-4ed0-b950-d193b2cb4759 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:30.349402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547da152-9890-41b4-8a3b-bc7a7ef2b5d0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:30.362164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326f8c5-bea1-45bd-943e-9e824b41e4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:30.364343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326f8c5-bea1-45bd-943e-9e824b41e4c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:30.364870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326f8c5-bea1-45bd-943e-9e824b41e4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:30.366015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326f8c5-bea1-45bd-943e-9e824b41e4c3 admin admin] PATH_INFO: `/v3/projects/79cf5c4a82b24f5a8dc242ef354dd321/users/37a5ae06dc9e4946bc3987a307cc3bb4/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:30.402023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326f8c5-bea1-45bd-943e-9e824b41e4c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79cf5c4a82b24f5a8dc242ef354dd321, user_id=37a5ae06dc9e4946bc3987a307cc3bb4, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:30.406157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326f8c5-bea1-45bd-943e-9e824b41e4c3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:30.412329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 836/1679] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:30.419341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5422c-64f2-4ed0-b950-d193b2cb4759 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:30.433748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 844/1680] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:30 2022] PUT /identity/v3/projects/79cf5c4a82b24f5a8dc242ef354dd321/users/37a5ae06dc9e4946bc3987a307cc3bb4/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:30.444216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68e63d-c0a1-4e25-a2a9-f5cc0bad549b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:30.446853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68e63d-c0a1-4e25-a2a9-f5cc0bad549b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:30.447345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68e63d-c0a1-4e25-a2a9-f5cc0bad549b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:30.482347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5422c-64f2-4ed0-b950-d193b2cb4759 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:30.512705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cc68e63d-c0a1-4e25-a2a9-f5cc0bad549b None None] MFA Rules not processed for user `37a5ae06dc9e4946bc3987a307cc3bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:30.517796 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cc68e63d-c0a1-4e25-a2a9-f5cc0bad549b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:30.673600 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 845/1681] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:30.674529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68e63d-c0a1-4e25-a2a9-f5cc0bad549b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:30.695419 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27acace-fb34-43bc-9f86-3f3cce62b5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:30.696877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27acace-fb34-43bc-9f86-3f3cce62b5aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:30.698909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27acace-fb34-43bc-9f86-3f3cce62b5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:30.699430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27acace-fb34-43bc-9f86-3f3cce62b5aa admin admin] PATH_INFO: `/v3/projects/aa97c5d804754d0bba9f925c422b2e3b/users/3dfe2b2fa7fa49399d21c877ce92f818/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:30.724324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27acace-fb34-43bc-9f86-3f3cce62b5aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa97c5d804754d0bba9f925c422b2e3b, user_id=3dfe2b2fa7fa49399d21c877ce92f818, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:30.731604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27acace-fb34-43bc-9f86-3f3cce62b5aa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:30.781248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 837/1682] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:30 2022] PUT /identity/v3/projects/aa97c5d804754d0bba9f925c422b2e3b/users/3dfe2b2fa7fa49399d21c877ce92f818/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:30.801081 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e3acc-4354-491e-ba63-e8bf62c77b57 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:30.806656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e3acc-4354-491e-ba63-e8bf62c77b57 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:30.808956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e3acc-4354-491e-ba63-e8bf62c77b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:30.842417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68e63d-c0a1-4e25-a2a9-f5cc0bad549b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:30.887684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:30.891376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c26e3acc-4354-491e-ba63-e8bf62c77b57 None None] MFA Rules not processed for user `3dfe2b2fa7fa49399d21c877ce92f818`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:30.892611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c26e3acc-4354-491e-ba63-e8bf62c77b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:31.045935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e48658-c14b-40a1-9a56-3e1610b7784c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:31.046623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e48658-c14b-40a1-9a56-3e1610b7784c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:31.046907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e48658-c14b-40a1-9a56-3e1610b7784c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:31.047182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e48658-c14b-40a1-9a56-3e1610b7784c admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:31.047959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e48658-c14b-40a1-9a56-3e1610b7784c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:31.059526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 838/1683] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:31.063366 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e48658-c14b-40a1-9a56-3e1610b7784c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:31.064071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2025-9784-4b72-9ede-6f11965197c3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:31.064367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2025-9784-4b72-9ede-6f11965197c3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:31.064651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2025-9784-4b72-9ede-6f11965197c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:31.121650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 846/1684] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:30 2022] POST /identity/v3/projects => generated 387 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:31.123821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e48658-c14b-40a1-9a56-3e1610b7784c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:31.226923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e48658-c14b-40a1-9a56-3e1610b7784c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:31.234703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b618d-431d-4596-a7fd-254fca501c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:31.236092 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b618d-431d-4596-a7fd-254fca501c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:31.236997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b618d-431d-4596-a7fd-254fca501c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:31.237479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b618d-431d-4596-a7fd-254fca501c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:31.239368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b618d-431d-4596-a7fd-254fca501c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:31.242430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b618d-431d-4596-a7fd-254fca501c32 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:31.332276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 847/1685] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:31 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:31.343471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b618d-431d-4596-a7fd-254fca501c32 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:31.471241 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0f8c2025-9784-4b72-9ede-6f11965197c3 None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:31.472825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0f8c2025-9784-4b72-9ede-6f11965197c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:31.474991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b618d-431d-4596-a7fd-254fca501c32 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:31.489119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36d42a-ceee-4a02-998c-18ad668ab161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:31.490443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36d42a-ceee-4a02-998c-18ad668ab161 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:31.490810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36d42a-ceee-4a02-998c-18ad668ab161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:31.491163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36d42a-ceee-4a02-998c-18ad668ab161 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:31.493065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36d42a-ceee-4a02-998c-18ad668ab161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:31.502678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36d42a-ceee-4a02-998c-18ad668ab161 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:31.569267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 848/1686] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:31 2022] POST /identity/v3/users => generated 300 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:31.569267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36d42a-ceee-4a02-998c-18ad668ab161 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:31.592399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36d42a-ceee-4a02-998c-18ad668ab161 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:31.599531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3b152-7983-43c5-9328-dc7632036ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:31.600256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3b152-7983-43c5-9328-dc7632036ffb admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:31.600541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3b152-7983-43c5-9328-dc7632036ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:31.600825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3b152-7983-43c5-9328-dc7632036ffb admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:31.601529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3b152-7983-43c5-9328-dc7632036ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:31.604075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3b152-7983-43c5-9328-dc7632036ffb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:31.618120 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 839/1687] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:20:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 558 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:31.620605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c2025-9784-4b72-9ede-6f11965197c3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:31.651729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:31.660524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 849/1688] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:31 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:31.664698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3b152-7983-43c5-9328-dc7632036ffb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:31.668061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c2025-9784-4b72-9ede-6f11965197c3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:31.679213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceda1c-7958-488d-bfbf-75c37f1245e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:31.679868 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceda1c-7958-488d-bfbf-75c37f1245e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:31.680158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceda1c-7958-488d-bfbf-75c37f1245e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:31.680435 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceda1c-7958-488d-bfbf-75c37f1245e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:31.681212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ceda1c-7958-488d-bfbf-75c37f1245e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:31.686043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3b152-7983-43c5-9328-dc7632036ffb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:31.689931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ceda1c-7958-488d-bfbf-75c37f1245e7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:31.701045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abffd6b-2f1b-4c79-9b9d-81d5de3a16d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:31.702988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abffd6b-2f1b-4c79-9b9d-81d5de3a16d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:31.705268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abffd6b-2f1b-4c79-9b9d-81d5de3a16d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:31.705820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abffd6b-2f1b-4c79-9b9d-81d5de3a16d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:31.707311 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:31.711900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abffd6b-2f1b-4c79-9b9d-81d5de3a16d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:31.718831 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 840/1689] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:31.726806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceda1c-7958-488d-bfbf-75c37f1245e7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:31.730321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abffd6b-2f1b-4c79-9b9d-81d5de3a16d3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:31.754817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceda1c-7958-488d-bfbf-75c37f1245e7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:31.768088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 850/1690] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:31.770155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abffd6b-2f1b-4c79-9b9d-81d5de3a16d3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:31.770286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15faac-f552-4f74-96ee-c294d895dc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:31.771236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15faac-f552-4f74-96ee-c294d895dc1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:31.771482 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15faac-f552-4f74-96ee-c294d895dc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:31.771765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15faac-f552-4f74-96ee-c294d895dc1f admin admin] PATH_INFO: `/v3/projects/71caaf817c824313a47912ee7d9f7140/users/719b51b3c6544b7986e919c6ea8943d6/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:31.796387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15faac-f552-4f74-96ee-c294d895dc1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=71caaf817c824313a47912ee7d9f7140, user_id=719b51b3c6544b7986e919c6ea8943d6, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:31.803198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abffd6b-2f1b-4c79-9b9d-81d5de3a16d3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:31.813808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15faac-f552-4f74-96ee-c294d895dc1f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:31.820044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d521ed-1e5b-478a-8e4c-87029a6f6648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:31.821279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d521ed-1e5b-478a-8e4c-87029a6f6648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:31.821820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d521ed-1e5b-478a-8e4c-87029a6f6648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:31.822891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d521ed-1e5b-478a-8e4c-87029a6f6648 admin admin] PATH_INFO: `/v3/projects/3eeb69d949684cde978970f704b4bce2/users/45ab78a822cc48d2880a794cab8995eb/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:31.857314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 841/1691] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:31 2022] PUT /identity/v3/projects/71caaf817c824313a47912ee7d9f7140/users/719b51b3c6544b7986e919c6ea8943d6/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:31.861407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15faac-f552-4f74-96ee-c294d895dc1f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:31.887905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d521ed-1e5b-478a-8e4c-87029a6f6648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eeb69d949684cde978970f704b4bce2, user_id=45ab78a822cc48d2880a794cab8995eb, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:31.892774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d521ed-1e5b-478a-8e4c-87029a6f6648 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:31.952670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 851/1692] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:31 2022] PUT /identity/v3/projects/3eeb69d949684cde978970f704b4bce2/users/45ab78a822cc48d2880a794cab8995eb/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:31.958080 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d521ed-1e5b-478a-8e4c-87029a6f6648 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:32.016948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15faac-f552-4f74-96ee-c294d895dc1f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:32.161954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d04bf-a3ed-43df-8471-a4fa575ba68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:32.163415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d04bf-a3ed-43df-8471-a4fa575ba68a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:32.163715 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d04bf-a3ed-43df-8471-a4fa575ba68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:32.164034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d04bf-a3ed-43df-8471-a4fa575ba68a admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:32.166248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d04bf-a3ed-43df-8471-a4fa575ba68a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:32.168169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d04bf-a3ed-43df-8471-a4fa575ba68a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:32.195784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 842/1693] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:32.195784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d04bf-a3ed-43df-8471-a4fa575ba68a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:32.201276 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0b8d04bf-a3ed-43df-8471-a4fa575ba68a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:32.244890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d521ed-1e5b-478a-8e4c-87029a6f6648 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:32.257837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3d149-5c15-4855-a550-d11b60a0409e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:32.260426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3d149-5c15-4855-a550-d11b60a0409e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:32.261086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3d149-5c15-4855-a550-d11b60a0409e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:32.262021 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3d149-5c15-4855-a550-d11b60a0409e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:32.263279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3d149-5c15-4855-a550-d11b60a0409e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:32.269588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3d149-5c15-4855-a550-d11b60a0409e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:32.308563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 852/1694] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:32.308563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3d149-5c15-4855-a550-d11b60a0409e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:32.334344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3d149-5c15-4855-a550-d11b60a0409e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:32.346869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573075ed-0f59-4efa-b3e1-90aa91e3da4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:32.347492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d04bf-a3ed-43df-8471-a4fa575ba68a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:32.350219 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573075ed-0f59-4efa-b3e1-90aa91e3da4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:32.350601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573075ed-0f59-4efa-b3e1-90aa91e3da4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:32.350888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573075ed-0f59-4efa-b3e1-90aa91e3da4c admin admin] PATH_INFO: `/v3/projects/71caaf817c824313a47912ee7d9f7140/users/719b51b3c6544b7986e919c6ea8943d6/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:32.357517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573075ed-0f59-4efa-b3e1-90aa91e3da4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71caaf817c824313a47912ee7d9f7140, user_id=719b51b3c6544b7986e919c6ea8943d6, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:32.363892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573075ed-0f59-4efa-b3e1-90aa91e3da4c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:32.366159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdeb738-8324-4a6f-a226-f08156414aa9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:32.366894 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeb738-8324-4a6f-a226-f08156414aa9 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:32.367189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeb738-8324-4a6f-a226-f08156414aa9 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:32.367470 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeb738-8324-4a6f-a226-f08156414aa9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:32.404104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 853/1695] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:32 2022] PUT /identity/v3/projects/71caaf817c824313a47912ee7d9f7140/users/719b51b3c6544b7986e919c6ea8943d6/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:32.405176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573075ed-0f59-4efa-b3e1-90aa91e3da4c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:32.429858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:32.597326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdeb738-8324-4a6f-a226-f08156414aa9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:32.598981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdeb738-8324-4a6f-a226-f08156414aa9 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:32.602701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573075ed-0f59-4efa-b3e1-90aa91e3da4c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:32.617504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647dbd6-27c9-4b60-9823-d04c68a0ff43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:32.618456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647dbd6-27c9-4b60-9823-d04c68a0ff43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:32.618741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647dbd6-27c9-4b60-9823-d04c68a0ff43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:32.619054 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647dbd6-27c9-4b60-9823-d04c68a0ff43 admin admin] PATH_INFO: `/v3/projects/3eeb69d949684cde978970f704b4bce2/users/45ab78a822cc48d2880a794cab8995eb/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:32.624282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647dbd6-27c9-4b60-9823-d04c68a0ff43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eeb69d949684cde978970f704b4bce2, user_id=45ab78a822cc48d2880a794cab8995eb, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:32.631847 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 843/1696] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:20:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:20:32.641178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16055a5-b090-4616-a923-4b1d19d8406f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:32.641789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647dbd6-27c9-4b60-9823-d04c68a0ff43 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:32.644722 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16055a5-b090-4616-a923-4b1d19d8406f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:32.645177 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16055a5-b090-4616-a923-4b1d19d8406f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:32.686877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 854/1697] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:32 2022] PUT /identity/v3/projects/3eeb69d949684cde978970f704b4bce2/users/45ab78a822cc48d2880a794cab8995eb/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:32.696840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae908cbb-ec21-48cc-89b6-b8f3fc25f754 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:32.698116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae908cbb-ec21-48cc-89b6-b8f3fc25f754 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:32.698401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae908cbb-ec21-48cc-89b6-b8f3fc25f754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:32.729058 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b16055a5-b090-4616-a923-4b1d19d8406f None None] MFA Rules not processed for user `719b51b3c6544b7986e919c6ea8943d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:32.746507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b16055a5-b090-4616-a923-4b1d19d8406f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:32.799498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ae908cbb-ec21-48cc-89b6-b8f3fc25f754 None None] MFA Rules not processed for user `45ab78a822cc48d2880a794cab8995eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:32.802460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ae908cbb-ec21-48cc-89b6-b8f3fc25f754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:32.872356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:32.974121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 855/1698] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:32.985290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae908cbb-ec21-48cc-89b6-b8f3fc25f754 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:32.987290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ae908cbb-ec21-48cc-89b6-b8f3fc25f754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:32.997682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 844/1699] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:33.003252 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16055a5-b090-4616-a923-4b1d19d8406f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:33.007264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b16055a5-b090-4616-a923-4b1d19d8406f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:33.158127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae908cbb-ec21-48cc-89b6-b8f3fc25f754 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:33.168636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d808432-d454-465b-8fe7-939c8dddb8f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:33.169586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d808432-d454-465b-8fe7-939c8dddb8f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:33.171363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d808432-d454-465b-8fe7-939c8dddb8f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:33.174097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d808432-d454-465b-8fe7-939c8dddb8f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:33.192119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d808432-d454-465b-8fe7-939c8dddb8f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:33.195051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d808432-d454-465b-8fe7-939c8dddb8f6 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:33.214770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16055a5-b090-4616-a923-4b1d19d8406f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:33.219773 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 856/1700] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:33.224797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81437951-1ac3-4ed6-b8ff-a906bcd5b147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:33.226253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437951-1ac3-4ed6-b8ff-a906bcd5b147 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:33.227220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437951-1ac3-4ed6-b8ff-a906bcd5b147 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:33.228280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437951-1ac3-4ed6-b8ff-a906bcd5b147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:33.251078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81437951-1ac3-4ed6-b8ff-a906bcd5b147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:33.253302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81437951-1ac3-4ed6-b8ff-a906bcd5b147 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:33.288363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 845/1701] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:33.320895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d808432-d454-465b-8fe7-939c8dddb8f6 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:33.410496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81437951-1ac3-4ed6-b8ff-a906bcd5b147 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:33.426528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81437951-1ac3-4ed6-b8ff-a906bcd5b147 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:33.439262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5a0c0-a4a6-4bbd-ac3e-109a4f4718f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:33.442456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5a0c0-a4a6-4bbd-ac3e-109a4f4718f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:33.442864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5a0c0-a4a6-4bbd-ac3e-109a4f4718f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:33.443252 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5a0c0-a4a6-4bbd-ac3e-109a4f4718f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:33.519356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:33.543505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5a0c0-a4a6-4bbd-ac3e-109a4f4718f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:33.545395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5a0c0-a4a6-4bbd-ac3e-109a4f4718f5 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:33.578802 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 846/1702] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:33.600999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d808432-d454-465b-8fe7-939c8dddb8f6 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:33.633638 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5cd67-0f35-4d55-9b55-4e404cc743fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:33.638827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5cd67-0f35-4d55-9b55-4e404cc743fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:33.640865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5cd67-0f35-4d55-9b55-4e404cc743fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:33.641173 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5cd67-0f35-4d55-9b55-4e404cc743fd admin admin] PATH_INFO: `/v3/users/37a5ae06dc9e4946bc3987a307cc3bb4` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:33.647215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5cd67-0f35-4d55-9b55-4e404cc743fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a5ae06dc9e4946bc3987a307cc3bb4)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:33.654422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5cd67-0f35-4d55-9b55-4e404cc743fd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:33.824688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4ae5cd67-0f35-4d55-9b55-4e404cc743fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a5ae06dc9e4946bc3987a307cc3bb4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:33.831237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4ae5cd67-0f35-4d55-9b55-4e404cc743fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a5ae06dc9e4946bc3987a307cc3bb4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:33.833933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4ae5cd67-0f35-4d55-9b55-4e404cc743fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a5ae06dc9e4946bc3987a307cc3bb4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:33.909396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4ae5cd67-0f35-4d55-9b55-4e404cc743fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a5ae06dc9e4946bc3987a307cc3bb4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:33.943661 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:33.957036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5a0c0-a4a6-4bbd-ac3e-109a4f4718f5 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:33.959080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-09c5a0c0-a4a6-4bbd-ac3e-109a4f4718f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:33.973152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4ae5cd67-0f35-4d55-9b55-4e404cc743fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a5ae06dc9e4946bc3987a307cc3bb4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:33.988741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5a0c0-a4a6-4bbd-ac3e-109a4f4718f5 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:34.000356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:34.001521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:34.001980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:34.002404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] PATH_INFO: `/v3/users/f858d1d451e44a0fbd3fdbff44220c0e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:34.007610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 857/1703] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:20:33 2022] DELETE /identity/v3/users/37a5ae06dc9e4946bc3987a307cc3bb4 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:34.011083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f858d1d451e44a0fbd3fdbff44220c0e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:34.012825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72f4e0-706a-4906-af32-b869fbdede10 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:34.013256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72f4e0-706a-4906-af32-b869fbdede10 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:34.013677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72f4e0-706a-4906-af32-b869fbdede10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:34.014719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:34.059947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7e72f4e0-706a-4906-af32-b869fbdede10 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:34.065230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7e72f4e0-706a-4906-af32-b869fbdede10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:34.173983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f858d1d451e44a0fbd3fdbff44220c0e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:34.188818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f858d1d451e44a0fbd3fdbff44220c0e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:34.195562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 858/1704] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:34.202689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f858d1d451e44a0fbd3fdbff44220c0e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:34.249983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f858d1d451e44a0fbd3fdbff44220c0e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:34.263491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f858d1d451e44a0fbd3fdbff44220c0e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:34.274230 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 847/1705] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:20:33 2022] DELETE /identity/v3/users/f858d1d451e44a0fbd3fdbff44220c0e => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:34.277328 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624ecf9-ac43-4990-aa08-d57c584f1e96 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:34.277328 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624ecf9-ac43-4990-aa08-d57c584f1e96 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:34.277546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624ecf9-ac43-4990-aa08-d57c584f1e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:34.300908 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f624ecf9-ac43-4990-aa08-d57c584f1e96 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:34.302739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f624ecf9-ac43-4990-aa08-d57c584f1e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:34.412958 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 859/1706] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:34.443323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:34.444961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:34.461454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624ecf9-ac43-4990-aa08-d57c584f1e96 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:34.463710 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f624ecf9-ac43-4990-aa08-d57c584f1e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:34.597562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ad3b5-68c0-437c-bbc6-0160e689075f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:34.616990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f874186-450b-407c-bc5f-6ff944bd5b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:34.618418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874186-450b-407c-bc5f-6ff944bd5b6a service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:34.619212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874186-450b-407c-bc5f-6ff944bd5b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:34.619709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874186-450b-407c-bc5f-6ff944bd5b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:34.674648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624ecf9-ac43-4990-aa08-d57c584f1e96 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:34.691480 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f24146-3fc3-445d-9b85-11fb46ed0972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:34.692306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f24146-3fc3-445d-9b85-11fb46ed0972 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:34.692596 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f24146-3fc3-445d-9b85-11fb46ed0972 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:34.692879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f24146-3fc3-445d-9b85-11fb46ed0972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:34.693106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f874186-450b-407c-bc5f-6ff944bd5b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:34.694986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f874186-450b-407c-bc5f-6ff944bd5b6a service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:34.701828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-01f24146-3fc3-445d-9b85-11fb46ed0972 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:34.716039 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 848/1707] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:34.718132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d44a4-7bb6-4d5c-9b02-79cd074697b4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:34.718789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d44a4-7bb6-4d5c-9b02-79cd074697b4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:34.720024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d44a4-7bb6-4d5c-9b02-79cd074697b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:34.768216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f24146-3fc3-445d-9b85-11fb46ed0972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:34.769689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f24146-3fc3-445d-9b85-11fb46ed0972 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:34.797944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 860/1708] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:34.814350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c45d44a4-7bb6-4d5c-9b02-79cd074697b4 None None] MFA Rules not processed for user `e42c796dbd7441289a7cbc328abeaf81`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:34.816186 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c45d44a4-7bb6-4d5c-9b02-79cd074697b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:34.859223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 849/1709] 10.222.0.84 () {58 vars in 1015 bytes} [Wed May 25 16:20:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:35.003365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f24146-3fc3-445d-9b85-11fb46ed0972 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:35.019528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f24146-3fc3-445d-9b85-11fb46ed0972 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:35.037130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33945ee7-8985-4615-8d1a-d52393b939c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:35.038554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33945ee7-8985-4615-8d1a-d52393b939c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:35.038978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33945ee7-8985-4615-8d1a-d52393b939c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:35.039454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33945ee7-8985-4615-8d1a-d52393b939c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:35.055708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33945ee7-8985-4615-8d1a-d52393b939c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:35.063843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33945ee7-8985-4615-8d1a-d52393b939c1 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:35.091465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 861/1710] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:20:37.487991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d44a4-7bb6-4d5c-9b02-79cd074697b4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:37.490274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c45d44a4-7bb6-4d5c-9b02-79cd074697b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:37.535319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d44a4-7bb6-4d5c-9b02-79cd074697b4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:37.558769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:37.559893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:37.560291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:37.560734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] PATH_INFO: `/v3/users/3dfe2b2fa7fa49399d21c877ce92f818` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:37.567071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dfe2b2fa7fa49399d21c877ce92f818)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:37.574789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:37.713925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dfe2b2fa7fa49399d21c877ce92f818'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:37.717726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:37.729734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dfe2b2fa7fa49399d21c877ce92f818'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:37.730866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dfe2b2fa7fa49399d21c877ce92f818'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:37.834641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dfe2b2fa7fa49399d21c877ce92f818'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:37.866044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dfe2b2fa7fa49399d21c877ce92f818'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:37.884124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 850/1711] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:20:37 2022] DELETE /identity/v3/users/3dfe2b2fa7fa49399d21c877ce92f818 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:37.889319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1361dfd-8654-46e9-b625-5e69e2562054 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:37.890271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1361dfd-8654-46e9-b625-5e69e2562054 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:37.890271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1361dfd-8654-46e9-b625-5e69e2562054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:37.944323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a1361dfd-8654-46e9-b625-5e69e2562054 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:37.945686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a1361dfd-8654-46e9-b625-5e69e2562054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:37.995156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:38.022696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:38.024432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:38.218161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63c2d8-eab6-47f6-8a08-23de98d491cb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:38.246949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 862/1712] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:38.248533 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1361dfd-8654-46e9-b625-5e69e2562054 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:38.252973 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a1361dfd-8654-46e9-b625-5e69e2562054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:38.268234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:38.270667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:38.272869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:38.276277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] PATH_INFO: `/v3/projects/79cf5c4a82b24f5a8dc242ef354dd321` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:38.280476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79cf5c4a82b24f5a8dc242ef354dd321)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:38.290638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:38.303329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1361dfd-8654-46e9-b625-5e69e2562054 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:38.317535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:38.318889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:38.319180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:38.319460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] PATH_INFO: `/v3/projects/8c0171bf802746578d0dd0267e0393d6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:38.321042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0171bf802746578d0dd0267e0393d6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:38.327525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:38.425132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79cf5c4a82b24f5a8dc242ef354dd321', 'request_id': 'req-ba43e98a-5d4c-43b5-afba-efbf2db69148'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.447309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79cf5c4a82b24f5a8dc242ef354dd321', 'request_id': 'req-ba43e98a-5d4c-43b5-afba-efbf2db69148'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.453755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0171bf802746578d0dd0267e0393d6', 'request_id': 'req-36301e20-eff6-4d59-a4c8-1915709ca676'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.470102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0171bf802746578d0dd0267e0393d6', 'request_id': 'req-36301e20-eff6-4d59-a4c8-1915709ca676'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.490188 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79cf5c4a82b24f5a8dc242ef354dd321', 'request_id': 'req-ba43e98a-5d4c-43b5-afba-efbf2db69148'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.496852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0171bf802746578d0dd0267e0393d6', 'request_id': 'req-36301e20-eff6-4d59-a4c8-1915709ca676'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.533181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] The token cache is being invalidate because project 79cf5c4a82b24f5a8dc242ef354dd321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:20:38.533477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.537143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.542990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] The token cache is being invalidate because project 8c0171bf802746578d0dd0267e0393d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:20:38.543294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.545410 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.551409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 851/1713] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:20:38 2022] DELETE /identity/v3/projects/79cf5c4a82b24f5a8dc242ef354dd321 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:38.551861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 863/1714] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:20:38 2022] DELETE /identity/v3/projects/8c0171bf802746578d0dd0267e0393d6 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:38.553671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:38.558505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:38.560941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:38.562150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:38.779781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43e98a-5d4c-43b5-afba-efbf2db69148 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:38.794958 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36301e20-eff6-4d59-a4c8-1915709ca676 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:38.804173 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:38.816999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e58533-2569-4adb-939d-378d3c24d85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:38.821546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e58533-2569-4adb-939d-378d3c24d85d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:38.822765 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e58533-2569-4adb-939d-378d3c24d85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:38.823064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e58533-2569-4adb-939d-378d3c24d85d admin admin] PATH_INFO: `/v3/users/719b51b3c6544b7986e919c6ea8943d6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:38.832334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e58533-2569-4adb-939d-378d3c24d85d admin admin] RBAC: Authorizing `identity:delete_user(user_id=719b51b3c6544b7986e919c6ea8943d6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:38.839286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e58533-2569-4adb-939d-378d3c24d85d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:38.922158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096615bc-89e5-423b-b173-b16a39ad8063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:38.922829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096615bc-89e5-423b-b173-b16a39ad8063 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:38.923112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096615bc-89e5-423b-b173-b16a39ad8063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:38.923408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096615bc-89e5-423b-b173-b16a39ad8063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:38.925303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-096615bc-89e5-423b-b173-b16a39ad8063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:38.948287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:38.950865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096615bc-89e5-423b-b173-b16a39ad8063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:38.951551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-00e58533-2569-4adb-939d-378d3c24d85d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '719b51b3c6544b7986e919c6ea8943d6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.952677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-00e58533-2569-4adb-939d-378d3c24d85d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '719b51b3c6544b7986e919c6ea8943d6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.954065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-00e58533-2569-4adb-939d-378d3c24d85d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '719b51b3c6544b7986e919c6ea8943d6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:38.958744 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096615bc-89e5-423b-b173-b16a39ad8063 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:38.980967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 852/1715] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:38.981876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096615bc-89e5-423b-b173-b16a39ad8063 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:38.990579 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-096615bc-89e5-423b-b173-b16a39ad8063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:39.001786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-00e58533-2569-4adb-939d-378d3c24d85d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '719b51b3c6544b7986e919c6ea8943d6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:39.010092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096615bc-89e5-423b-b173-b16a39ad8063 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:39.027076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:39.027966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:39.028239 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:39.028516 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] PATH_INFO: `/v3/users/83c8dd0277cb452ca56e0d5a50ed0cbd` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:39.032582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=83c8dd0277cb452ca56e0d5a50ed0cbd)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:39.034698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:39.035630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-00e58533-2569-4adb-939d-378d3c24d85d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '719b51b3c6544b7986e919c6ea8943d6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:39.083940 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 864/1716] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:20:38 2022] DELETE /identity/v3/users/719b51b3c6544b7986e919c6ea8943d6 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:39.088778 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b3d4c-0a82-462c-95a4-2e03eb383a48 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:39.090423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b3d4c-0a82-462c-95a4-2e03eb383a48 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:39.091445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b3d4c-0a82-462c-95a4-2e03eb383a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:39.135091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83c8dd0277cb452ca56e0d5a50ed0cbd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:39.153040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83c8dd0277cb452ca56e0d5a50ed0cbd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:39.153739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83c8dd0277cb452ca56e0d5a50ed0cbd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:39.174720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83c8dd0277cb452ca56e0d5a50ed0cbd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:39.190199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83c8dd0277cb452ca56e0d5a50ed0cbd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:39.192228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c92b3d4c-0a82-462c-95a4-2e03eb383a48 None None] MFA Rules not processed for user `15fae427f22d476fa0e378949de41292`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:39.193509 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c92b3d4c-0a82-462c-95a4-2e03eb383a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:39.213791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 853/1717] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:20:38 2022] DELETE /identity/v3/users/83c8dd0277cb452ca56e0d5a50ed0cbd => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:39.490777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 865/1718] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:39.500745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:39.502989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:39.669313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26402f-f54f-441f-afde-0bf6f98638eb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:39.678491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e2d7b-3332-46d9-8569-20c80b6c99e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:39.679180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e2d7b-3332-46d9-8569-20c80b6c99e2 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:39.679462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e2d7b-3332-46d9-8569-20c80b6c99e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:39.679738 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e2d7b-3332-46d9-8569-20c80b6c99e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:39.698748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e2d7b-3332-46d9-8569-20c80b6c99e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:39.700403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e2d7b-3332-46d9-8569-20c80b6c99e2 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:39.732250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 854/1719] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:39.920584 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522d91-41c0-4380-b4da-3aca92787aef None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:39.921038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522d91-41c0-4380-b4da-3aca92787aef None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:39.922026 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522d91-41c0-4380-b4da-3aca92787aef None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:39.937446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 866/1720] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:20:39 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:20:39.971579 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6cacf-5b7a-4b35-8985-f99dbf77e87f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:39.971983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6cacf-5b7a-4b35-8985-f99dbf77e87f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:39.972414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6cacf-5b7a-4b35-8985-f99dbf77e87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:40.012854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4fb6cacf-5b7a-4b35-8985-f99dbf77e87f None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:40.018642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4fb6cacf-5b7a-4b35-8985-f99dbf77e87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:40.045963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 855/1721] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:20:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:40.080596 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52522d91-41c0-4380-b4da-3aca92787aef None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:40.082653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-52522d91-41c0-4380-b4da-3aca92787aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:40.130326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52522d91-41c0-4380-b4da-3aca92787aef None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:40.140299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b4462-6578-43a4-8dda-003f9f6e8f1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:40.141188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b4462-6578-43a4-8dda-003f9f6e8f1b service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:40.141636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b4462-6578-43a4-8dda-003f9f6e8f1b service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:40.142041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b4462-6578-43a4-8dda-003f9f6e8f1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:40.170695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b4462-6578-43a4-8dda-003f9f6e8f1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:40.172448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b4462-6578-43a4-8dda-003f9f6e8f1b service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:40.204253 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 867/1722] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:40.865061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded83c9-f544-43fa-a382-5975d256d24c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:40.865667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded83c9-f544-43fa-a382-5975d256d24c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:40.866035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded83c9-f544-43fa-a382-5975d256d24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:40.895544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:40.935997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9ded83c9-f544-43fa-a382-5975d256d24c None None] MFA Rules not processed for user `30de794b71b343289491531dfbe37f53`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:40.937361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ded83c9-f544-43fa-a382-5975d256d24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:41.114521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 856/1723] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:41.121861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4d1c8-e0ee-40ec-bb45-06a3667dd5db None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:41.122357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4d1c8-e0ee-40ec-bb45-06a3667dd5db None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:41.122671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4d1c8-e0ee-40ec-bb45-06a3667dd5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:41.158123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9ad4d1c8-e0ee-40ec-bb45-06a3667dd5db None None] MFA Rules not processed for user `30de794b71b343289491531dfbe37f53`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:41.160103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded83c9-f544-43fa-a382-5975d256d24c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:41.163233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ded83c9-f544-43fa-a382-5975d256d24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:41.177098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9ad4d1c8-e0ee-40ec-bb45-06a3667dd5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:41.247657 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 868/1724] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:41.252316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4d1c8-e0ee-40ec-bb45-06a3667dd5db None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:41.300247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4d1c8-e0ee-40ec-bb45-06a3667dd5db None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:41.336384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3159fc4-f41a-4769-8b2b-9c83a039b70d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:41.337093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3159fc4-f41a-4769-8b2b-9c83a039b70d service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:41.338360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3159fc4-f41a-4769-8b2b-9c83a039b70d service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:41.338490 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3159fc4-f41a-4769-8b2b-9c83a039b70d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:41.381636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3159fc4-f41a-4769-8b2b-9c83a039b70d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:41.383331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3159fc4-f41a-4769-8b2b-9c83a039b70d service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:41.420475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 869/1725] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:41.459663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded83c9-f544-43fa-a382-5975d256d24c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:41.474376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:41.476025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:41.476499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:41.476923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] PATH_INFO: `/v3/projects/37de430b0d774c1799e70a67cbf07db4` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:41.478812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=37de430b0d774c1799e70a67cbf07db4)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:41.486164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:41.662287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37de430b0d774c1799e70a67cbf07db4', 'request_id': 'req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:41.675454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37de430b0d774c1799e70a67cbf07db4', 'request_id': 'req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:41.686292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37de430b0d774c1799e70a67cbf07db4', 'request_id': 'req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:41.728619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] The token cache is being invalidate because project 37de430b0d774c1799e70a67cbf07db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:20:41.729263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:41.730124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-764f81fc-f74c-40e0-9cf0-cf52ae3d2fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:41.735805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 857/1726] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:20:41 2022] DELETE /identity/v3/projects/37de430b0d774c1799e70a67cbf07db4 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:41.737288 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547823cb-ca01-494d-a39c-401fc08bdfc3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:41.737741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547823cb-ca01-494d-a39c-401fc08bdfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:41.738117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547823cb-ca01-494d-a39c-401fc08bdfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:41.782079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-547823cb-ca01-494d-a39c-401fc08bdfc3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:41.783562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-547823cb-ca01-494d-a39c-401fc08bdfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:41.896883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 870/1727] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:41.898734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a5fc8-3c15-4d28-812d-43a6bcbeec15 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:41.899572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a5fc8-3c15-4d28-812d-43a6bcbeec15 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:41.900961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a5fc8-3c15-4d28-812d-43a6bcbeec15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:41.947398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b96963-af73-4acc-8635-011da51396b9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:41.947879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b96963-af73-4acc-8635-011da51396b9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:41.948272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b96963-af73-4acc-8635-011da51396b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:41.980925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-317a5fc8-3c15-4d28-812d-43a6bcbeec15 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:41.984210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-317a5fc8-3c15-4d28-812d-43a6bcbeec15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:42.001446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-92b96963-af73-4acc-8635-011da51396b9 None None] MFA Rules not processed for user `b781c4c6cb61439fb781988a7da028ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:42.007735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-92b96963-af73-4acc-8635-011da51396b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:42.022138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 858/1728] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:20:42.028969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a5fc8-3c15-4d28-812d-43a6bcbeec15 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:42.063513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a5fc8-3c15-4d28-812d-43a6bcbeec15 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:42.067001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:42.091536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870b98b-c557-4ec7-9681-9ff4598e3a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:42.092439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870b98b-c557-4ec7-9681-9ff4598e3a6e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:42.097144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870b98b-c557-4ec7-9681-9ff4598e3a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:42.097306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870b98b-c557-4ec7-9681-9ff4598e3a6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:42.099614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870b98b-c557-4ec7-9681-9ff4598e3a6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:42.099798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870b98b-c557-4ec7-9681-9ff4598e3a6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:42.113973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870b98b-c557-4ec7-9681-9ff4598e3a6e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:42.184180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 859/1729] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:42.185226 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870b98b-c557-4ec7-9681-9ff4598e3a6e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:42.234386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870b98b-c557-4ec7-9681-9ff4598e3a6e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:42.261467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a3110-94fb-49c4-be9b-326b43e87e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:42.262337 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a3110-94fb-49c4-be9b-326b43e87e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:42.262473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a3110-94fb-49c4-be9b-326b43e87e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:42.262752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a3110-94fb-49c4-be9b-326b43e87e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:42.263454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a3110-94fb-49c4-be9b-326b43e87e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:42.265417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a3110-94fb-49c4-be9b-326b43e87e00 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:42.299380 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:42.301499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 871/1730] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:42.428921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 860/1731] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:42 2022] POST /identity/v3/projects => generated 365 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:42.433545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b96963-af73-4acc-8635-011da51396b9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:42.449488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b96963-af73-4acc-8635-011da51396b9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:42.657958 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c779c22-072c-4148-87cb-f1fa5b619ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:42.658685 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c779c22-072c-4148-87cb-f1fa5b619ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:42.658983 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c779c22-072c-4148-87cb-f1fa5b619ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:42.659310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c779c22-072c-4148-87cb-f1fa5b619ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:42.664506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c779c22-072c-4148-87cb-f1fa5b619ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:42.667613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c779c22-072c-4148-87cb-f1fa5b619ec7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:42.718822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 872/1732] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:42 2022] POST /identity/v3/users => generated 283 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:42.722181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a3110-94fb-49c4-be9b-326b43e87e00 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:42.738735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a3110-94fb-49c4-be9b-326b43e87e00 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:42.754499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063645a7-d93b-47f8-98ac-d072c312d64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:42.755497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063645a7-d93b-47f8-98ac-d072c312d64e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:42.755985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063645a7-d93b-47f8-98ac-d072c312d64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:42.756416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063645a7-d93b-47f8-98ac-d072c312d64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:42.758229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063645a7-d93b-47f8-98ac-d072c312d64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:42.764293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063645a7-d93b-47f8-98ac-d072c312d64e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:42.796122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 861/1733] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:42.800848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c779c22-072c-4148-87cb-f1fa5b619ec7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:42.827798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c779c22-072c-4148-87cb-f1fa5b619ec7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:42.847722 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790bf2d-2505-434c-b006-d53720a1ee40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:42.847722 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790bf2d-2505-434c-b006-d53720a1ee40 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:42.847722 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790bf2d-2505-434c-b006-d53720a1ee40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:42.847722 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790bf2d-2505-434c-b006-d53720a1ee40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:42.848095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790bf2d-2505-434c-b006-d53720a1ee40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:42.849026 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063645a7-d93b-47f8-98ac-d072c312d64e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:42.850033 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790bf2d-2505-434c-b006-d53720a1ee40 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:42.851056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-063645a7-d93b-47f8-98ac-d072c312d64e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:42.877495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 873/1734] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:20:42.880677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790bf2d-2505-434c-b006-d53720a1ee40 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:42.893764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063645a7-d93b-47f8-98ac-d072c312d64e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:42.903988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790bf2d-2505-434c-b006-d53720a1ee40 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:42.909639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab7ade-485d-4609-9d1d-3365f23f66e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:42.911265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab7ade-485d-4609-9d1d-3365f23f66e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:42.911599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab7ade-485d-4609-9d1d-3365f23f66e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:42.912095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab7ade-485d-4609-9d1d-3365f23f66e0 admin admin] PATH_INFO: `/v3/projects/1eee3cafb6d1424b81a6c543a48fe275/users/e0960f850d38472b895c150bd8c5ce87/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:42.919647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:42.920556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:42.920877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:42.921149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] PATH_INFO: `/v3/projects/71caaf817c824313a47912ee7d9f7140` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:42.928422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=71caaf817c824313a47912ee7d9f7140)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:42.939627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:42.952629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab7ade-485d-4609-9d1d-3365f23f66e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eee3cafb6d1424b81a6c543a48fe275, user_id=e0960f850d38472b895c150bd8c5ce87, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:42.956160 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab7ade-485d-4609-9d1d-3365f23f66e0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:43.017083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 874/1735] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:42 2022] PUT /identity/v3/projects/1eee3cafb6d1424b81a6c543a48fe275/users/e0960f850d38472b895c150bd8c5ce87/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:43.017083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab7ade-485d-4609-9d1d-3365f23f66e0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:43.017927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c1ab7ade-485d-4609-9d1d-3365f23f66e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:43.067864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71caaf817c824313a47912ee7d9f7140', 'request_id': 'req-751f6176-f618-4080-b3bd-744310ccf2ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:43.080550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71caaf817c824313a47912ee7d9f7140', 'request_id': 'req-751f6176-f618-4080-b3bd-744310ccf2ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:43.093202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71caaf817c824313a47912ee7d9f7140', 'request_id': 'req-751f6176-f618-4080-b3bd-744310ccf2ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:43.120056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] The token cache is being invalidate because project 71caaf817c824313a47912ee7d9f7140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:20:43.120551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:43.123489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-751f6176-f618-4080-b3bd-744310ccf2ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:43.130273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 862/1736] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:20:42 2022] DELETE /identity/v3/projects/71caaf817c824313a47912ee7d9f7140 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:20:43.144167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ea0c-1822-4ccc-a1f7-bf5d409ca48b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:43.145844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ea0c-1822-4ccc-a1f7-bf5d409ca48b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:43.146396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ea0c-1822-4ccc-a1f7-bf5d409ca48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:43.201613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab7ade-485d-4609-9d1d-3365f23f66e0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:43.243712 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:43.244784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9f20ea0c-1822-4ccc-a1f7-bf5d409ca48b None None] MFA Rules not processed for user `e0960f850d38472b895c150bd8c5ce87`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:43.254231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9f20ea0c-1822-4ccc-a1f7-bf5d409ca48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:43.362816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:43.363714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:43.364013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:43.364276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] PATH_INFO: `/v3/projects/aa97c5d804754d0bba9f925c422b2e3b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:43.365956 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa97c5d804754d0bba9f925c422b2e3b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:43.367952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:43.463984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 863/1737] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:43.465408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dd9c9-d674-4052-863e-2fa68c5bcea3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:43.465696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dd9c9-d674-4052-863e-2fa68c5bcea3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:43.465995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dd9c9-d674-4052-863e-2fa68c5bcea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:43.531768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa97c5d804754d0bba9f925c422b2e3b', 'request_id': 'req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:43.534865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d63dd9c9-d674-4052-863e-2fa68c5bcea3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:43.536120 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d63dd9c9-d674-4052-863e-2fa68c5bcea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:43.548432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa97c5d804754d0bba9f925c422b2e3b', 'request_id': 'req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:43.576701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa97c5d804754d0bba9f925c422b2e3b', 'request_id': 'req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:43.601532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] The token cache is being invalidate because project aa97c5d804754d0bba9f925c422b2e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:20:43.603648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:43.604663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6df887ba-8925-4b63-b8fb-f9aaa84a3a4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:43.609413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 875/1738] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:20:43 2022] DELETE /identity/v3/projects/aa97c5d804754d0bba9f925c422b2e3b => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:43.613307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017ebff-68c8-4898-ac54-1c1dad981228 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:43.613608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017ebff-68c8-4898-ac54-1c1dad981228 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:43.615726 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017ebff-68c8-4898-ac54-1c1dad981228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:43.659050 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:43.707409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6017ebff-68c8-4898-ac54-1c1dad981228 None None] MFA Rules not processed for user `9178e89a790d48d19b25c4c22bd0ca4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:43.716773 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6017ebff-68c8-4898-ac54-1c1dad981228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:43.725152 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 864/1739] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:43.726075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dd9c9-d674-4052-863e-2fa68c5bcea3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:43.733010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d63dd9c9-d674-4052-863e-2fa68c5bcea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:43.788744 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dd9c9-d674-4052-863e-2fa68c5bcea3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:43.799169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba83ee-c389-4e30-b77a-a530a3714950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:43.799841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba83ee-c389-4e30-b77a-a530a3714950 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:43.800778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba83ee-c389-4e30-b77a-a530a3714950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:43.801080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba83ee-c389-4e30-b77a-a530a3714950 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:43.802027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba83ee-c389-4e30-b77a-a530a3714950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:43.803962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba83ee-c389-4e30-b77a-a530a3714950 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:43.915617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 865/1740] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:43 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:43.916798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba83ee-c389-4e30-b77a-a530a3714950 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:43.923292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c9ba83ee-c389-4e30-b77a-a530a3714950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:43.924093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 876/1741] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:43.928405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7b273-ea7f-413a-a215-00a53827adc2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:43.928702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7b273-ea7f-413a-a215-00a53827adc2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:43.933795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7b273-ea7f-413a-a215-00a53827adc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:43.986752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3ed7b273-ea7f-413a-a215-00a53827adc2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:43.988173 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ed7b273-ea7f-413a-a215-00a53827adc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:44.139569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:44.141318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba83ee-c389-4e30-b77a-a530a3714950 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:44.159669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5496480-4657-41e2-af1a-c0e45ac3e0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:44.164961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5496480-4657-41e2-af1a-c0e45ac3e0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:44.165837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5496480-4657-41e2-af1a-c0e45ac3e0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:44.166117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5496480-4657-41e2-af1a-c0e45ac3e0eb admin admin] PATH_INFO: `/v3/users/45ab78a822cc48d2880a794cab8995eb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:44.170791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5496480-4657-41e2-af1a-c0e45ac3e0eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ab78a822cc48d2880a794cab8995eb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:44.172786 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5496480-4657-41e2-af1a-c0e45ac3e0eb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:44.200026 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 877/1742] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:44.206703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181977b-5f94-49da-b59c-fd694cfa133a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:44.208417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181977b-5f94-49da-b59c-fd694cfa133a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:44.208914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181977b-5f94-49da-b59c-fd694cfa133a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:44.210623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 878/1743] 10.222.0.42 () {54 vars in 912 bytes} [Wed May 25 16:20:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:20:44.212906 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181977b-5f94-49da-b59c-fd694cfa133a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:44.246601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181977b-5f94-49da-b59c-fd694cfa133a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:44.259948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728c899-18b4-4d7c-94c1-483612bb1db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:44.260764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728c899-18b4-4d7c-94c1-483612bb1db0 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:44.261153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728c899-18b4-4d7c-94c1-483612bb1db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:44.273677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728c899-18b4-4d7c-94c1-483612bb1db0 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:44.276589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728c899-18b4-4d7c-94c1-483612bb1db0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:44.283660 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728c899-18b4-4d7c-94c1-483612bb1db0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:44.358724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f5496480-4657-41e2-af1a-c0e45ac3e0eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ab78a822cc48d2880a794cab8995eb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:44.365817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 879/1744] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:44 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:44.371548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728c899-18b4-4d7c-94c1-483612bb1db0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:44.402105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f5496480-4657-41e2-af1a-c0e45ac3e0eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ab78a822cc48d2880a794cab8995eb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:44.406739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f5496480-4657-41e2-af1a-c0e45ac3e0eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ab78a822cc48d2880a794cab8995eb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:44.428384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728c899-18b4-4d7c-94c1-483612bb1db0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:44.462248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f5496480-4657-41e2-af1a-c0e45ac3e0eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ab78a822cc48d2880a794cab8995eb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:44.465891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca7914-9f5a-4961-a231-d8d1c5dedf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:44.466959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca7914-9f5a-4961-a231-d8d1c5dedf79 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:44.467470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca7914-9f5a-4961-a231-d8d1c5dedf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:44.468046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca7914-9f5a-4961-a231-d8d1c5dedf79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:44.469010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca7914-9f5a-4961-a231-d8d1c5dedf79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:44.469538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca7914-9f5a-4961-a231-d8d1c5dedf79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:44.471685 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca7914-9f5a-4961-a231-d8d1c5dedf79 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:44.484097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f5496480-4657-41e2-af1a-c0e45ac3e0eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ab78a822cc48d2880a794cab8995eb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:44.506062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 866/1745] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:20:43 2022] DELETE /identity/v3/users/45ab78a822cc48d2880a794cab8995eb => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:44.507637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a28b4-7b89-4897-bd20-359cc97aebdc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:44.507851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a28b4-7b89-4897-bd20-359cc97aebdc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:44.510959 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a28b4-7b89-4897-bd20-359cc97aebdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:44.528928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 880/1746] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:44.530419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca7914-9f5a-4961-a231-d8d1c5dedf79 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:44.534562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-40ca7914-9f5a-4961-a231-d8d1c5dedf79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:44.567744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca7914-9f5a-4961-a231-d8d1c5dedf79 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:44.569138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c15a28b4-7b89-4897-bd20-359cc97aebdc None None] MFA Rules not processed for user `e42c796dbd7441289a7cbc328abeaf81`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:44.571348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c15a28b4-7b89-4897-bd20-359cc97aebdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:44.752689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb32da-ad1c-4602-9d20-7327c965e4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:44.753542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb32da-ad1c-4602-9d20-7327c965e4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:44.754019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb32da-ad1c-4602-9d20-7327c965e4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:44.754496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb32da-ad1c-4602-9d20-7327c965e4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:44.755207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:44.755447 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb32da-ad1c-4602-9d20-7327c965e4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:44.757217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb32da-ad1c-4602-9d20-7327c965e4c5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:44.790857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 881/1747] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:44.794667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb32da-ad1c-4602-9d20-7327c965e4c5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:44.802217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e8bb32da-ad1c-4602-9d20-7327c965e4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:44.824903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 867/1748] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:20:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:44.826096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a28b4-7b89-4897-bd20-359cc97aebdc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:44.856214 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb32da-ad1c-4602-9d20-7327c965e4c5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:44.875721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a28b4-7b89-4897-bd20-359cc97aebdc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:44.892649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd6ab4-bc82-4168-b7fd-8ca13e5beee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:44.894161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd6ab4-bc82-4168-b7fd-8ca13e5beee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:44.895608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd6ab4-bc82-4168-b7fd-8ca13e5beee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:44.896533 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fca34-7367-4eb2-9cc6-103aa22b027c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:44.896978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd6ab4-bc82-4168-b7fd-8ca13e5beee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:44.898904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fca34-7367-4eb2-9cc6-103aa22b027c admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:44.899233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd6ab4-bc82-4168-b7fd-8ca13e5beee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:44.901704 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd6ab4-bc82-4168-b7fd-8ca13e5beee9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:44.902161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fca34-7367-4eb2-9cc6-103aa22b027c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:44.902567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fca34-7367-4eb2-9cc6-103aa22b027c admin admin] PATH_INFO: `/v3/projects/16b3b731b63f439daca87e778bd0747a/users/df137d2d7ede4e368ca524adc85ab72c/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:44.947159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fca34-7367-4eb2-9cc6-103aa22b027c admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b3b731b63f439daca87e778bd0747a, user_id=df137d2d7ede4e368ca524adc85ab72c, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:44.952818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fca34-7367-4eb2-9cc6-103aa22b027c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:44.996866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 882/1749] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:44 2022] POST /identity/v3/projects => generated 387 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:45.001489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd6ab4-bc82-4168-b7fd-8ca13e5beee9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:45.014708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-19bd6ab4-bc82-4168-b7fd-8ca13e5beee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:45.029810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 868/1750] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:44 2022] PUT /identity/v3/projects/16b3b731b63f439daca87e778bd0747a/users/df137d2d7ede4e368ca524adc85ab72c/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:45.033141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fca34-7367-4eb2-9cc6-103aa22b027c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:45.227070 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd6ab4-bc82-4168-b7fd-8ca13e5beee9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:45.247436 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fca34-7367-4eb2-9cc6-103aa22b027c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:45.268063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220bdc4-38c9-4812-8fd9-630bf17685d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:45.269036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220bdc4-38c9-4812-8fd9-630bf17685d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:45.271435 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220bdc4-38c9-4812-8fd9-630bf17685d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:45.272300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220bdc4-38c9-4812-8fd9-630bf17685d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:45.275179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220bdc4-38c9-4812-8fd9-630bf17685d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:45.280213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220bdc4-38c9-4812-8fd9-630bf17685d6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:45.337351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 869/1751] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:45 2022] POST /identity/v3/users => generated 294 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:45.342031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220bdc4-38c9-4812-8fd9-630bf17685d6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:45.359183 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:45.362781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220bdc4-38c9-4812-8fd9-630bf17685d6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:45.375749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a4549-a80d-434a-af21-1689f22677ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:45.378352 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a4549-a80d-434a-af21-1689f22677ee admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:45.378663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9ead7-d90f-4710-8894-8b3b50636e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:45.379906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a4549-a80d-434a-af21-1689f22677ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:45.379906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a4549-a80d-434a-af21-1689f22677ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:45.380319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9ead7-d90f-4710-8894-8b3b50636e08 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:45.380961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9ead7-d90f-4710-8894-8b3b50636e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:45.382071 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9ead7-d90f-4710-8894-8b3b50636e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:45.382807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a4549-a80d-434a-af21-1689f22677ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:45.386310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a4549-a80d-434a-af21-1689f22677ee admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:45.402835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9ead7-d90f-4710-8894-8b3b50636e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:45.408824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9ead7-d90f-4710-8894-8b3b50636e08 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:45.416464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 870/1752] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:45.419313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a4549-a80d-434a-af21-1689f22677ee admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:45.436268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 883/1753] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:45.439161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9ead7-d90f-4710-8894-8b3b50636e08 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:45.452956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a4549-a80d-434a-af21-1689f22677ee admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:45.479190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9ead7-d90f-4710-8894-8b3b50636e08 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:45.480204 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c801081-2882-4ec0-a1dc-ca2c02cb84e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:45.481139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c801081-2882-4ec0-a1dc-ca2c02cb84e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:45.481646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c801081-2882-4ec0-a1dc-ca2c02cb84e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:45.482168 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c801081-2882-4ec0-a1dc-ca2c02cb84e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:45.483124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c801081-2882-4ec0-a1dc-ca2c02cb84e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:45.485008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c801081-2882-4ec0-a1dc-ca2c02cb84e2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:45.498885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a1729-ae73-49a7-995e-80f263efc839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:45.500318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a1729-ae73-49a7-995e-80f263efc839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:45.500828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a1729-ae73-49a7-995e-80f263efc839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:45.501416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a1729-ae73-49a7-995e-80f263efc839 admin admin] PATH_INFO: `/v3/projects/16b3b731b63f439daca87e778bd0747a/users/df137d2d7ede4e368ca524adc85ab72c/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:45.506934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a1729-ae73-49a7-995e-80f263efc839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b3b731b63f439daca87e778bd0747a, user_id=df137d2d7ede4e368ca524adc85ab72c, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:45.510800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a1729-ae73-49a7-995e-80f263efc839 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:45.517897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 871/1754] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:45.519449 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c801081-2882-4ec0-a1dc-ca2c02cb84e2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:45.539071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c801081-2882-4ec0-a1dc-ca2c02cb84e2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:45.546352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 884/1755] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:45 2022] PUT /identity/v3/projects/16b3b731b63f439daca87e778bd0747a/users/df137d2d7ede4e368ca524adc85ab72c/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:45.559292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b68f-785c-4107-835c-dc23ebe0c9f9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:45.560348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b68f-785c-4107-835c-dc23ebe0c9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:45.563040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b68f-785c-4107-835c-dc23ebe0c9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:45.640353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-da90b68f-785c-4107-835c-dc23ebe0c9f9 None None] MFA Rules not processed for user `df137d2d7ede4e368ca524adc85ab72c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:45.643585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-da90b68f-785c-4107-835c-dc23ebe0c9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:45.711087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6471b8d-89a3-4831-9a81-a49387e7d3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:45.711966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6471b8d-89a3-4831-9a81-a49387e7d3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:45.712604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6471b8d-89a3-4831-9a81-a49387e7d3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:45.713002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6471b8d-89a3-4831-9a81-a49387e7d3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:45.714079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6471b8d-89a3-4831-9a81-a49387e7d3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:45.715871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6471b8d-89a3-4831-9a81-a49387e7d3f5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:45.755947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 872/1756] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:45.757358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6471b8d-89a3-4831-9a81-a49387e7d3f5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:45.798823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6471b8d-89a3-4831-9a81-a49387e7d3f5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:45.819456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990196e0-e705-4fa3-852d-9d27be97cd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:45.820776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990196e0-e705-4fa3-852d-9d27be97cd58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:45.822264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990196e0-e705-4fa3-852d-9d27be97cd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:45.824840 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990196e0-e705-4fa3-852d-9d27be97cd58 admin admin] PATH_INFO: `/v3/projects/b1b0077769d14ca6bf6922533bf27952/users/69a6c9ca9018422bbda4ffcad74716ec/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:45.842518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 885/1757] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:45.860345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90b68f-785c-4107-835c-dc23ebe0c9f9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:45.861922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-da90b68f-785c-4107-835c-dc23ebe0c9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:45.896233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990196e0-e705-4fa3-852d-9d27be97cd58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b0077769d14ca6bf6922533bf27952, user_id=69a6c9ca9018422bbda4ffcad74716ec, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:45.900186 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990196e0-e705-4fa3-852d-9d27be97cd58 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:45.960907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 873/1758] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:45 2022] PUT /identity/v3/projects/b1b0077769d14ca6bf6922533bf27952/users/69a6c9ca9018422bbda4ffcad74716ec/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:45.964187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32a89-501f-4414-bd52-c4649213153e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:45.965391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32a89-501f-4414-bd52-c4649213153e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:45.966614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32a89-501f-4414-bd52-c4649213153e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:46.046785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:46.071189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-daf32a89-501f-4414-bd52-c4649213153e None None] MFA Rules not processed for user `69a6c9ca9018422bbda4ffcad74716ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:46.075122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-daf32a89-501f-4414-bd52-c4649213153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:46.083756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90b68f-785c-4107-835c-dc23ebe0c9f9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:46.298599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39817b84-d81e-411f-91aa-1fdfb2108614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:46.304576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39817b84-d81e-411f-91aa-1fdfb2108614 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:46.305046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39817b84-d81e-411f-91aa-1fdfb2108614 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:46.305280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39817b84-d81e-411f-91aa-1fdfb2108614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:46.332497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 874/1759] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:46.338639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf32a89-501f-4414-bd52-c4649213153e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:46.455579 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:46.568237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39817b84-d81e-411f-91aa-1fdfb2108614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:46.578241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39817b84-d81e-411f-91aa-1fdfb2108614 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:46.614232 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 886/1760] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 754 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:20:46.615589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39817b84-d81e-411f-91aa-1fdfb2108614 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:46.627671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-39817b84-d81e-411f-91aa-1fdfb2108614 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:46.639418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf32a89-501f-4414-bd52-c4649213153e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:46.645956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834c107-69d0-41a8-8bb6-790a1daa8800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:46.646651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c107-69d0-41a8-8bb6-790a1daa8800 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:46.646933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c107-69d0-41a8-8bb6-790a1daa8800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:46.647212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c107-69d0-41a8-8bb6-790a1daa8800 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:46.647948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834c107-69d0-41a8-8bb6-790a1daa8800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:46.658584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834c107-69d0-41a8-8bb6-790a1daa8800 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:46.691243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39817b84-d81e-411f-91aa-1fdfb2108614 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:46.713617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:46.714413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:46.714639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:46.714857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] PATH_INFO: `/v3/projects/3eeb69d949684cde978970f704b4bce2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:46.721078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eeb69d949684cde978970f704b4bce2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:46.723185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:46.828749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 875/1761] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:46 2022] POST /identity/v3/projects => generated 389 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:46.829956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834c107-69d0-41a8-8bb6-790a1daa8800 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:46.874964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eeb69d949684cde978970f704b4bce2', 'request_id': 'req-887a3a47-1534-4856-83c6-77c96464b99a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:46.893189 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eeb69d949684cde978970f704b4bce2', 'request_id': 'req-887a3a47-1534-4856-83c6-77c96464b99a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:46.920058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eeb69d949684cde978970f704b4bce2', 'request_id': 'req-887a3a47-1534-4856-83c6-77c96464b99a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:46.980561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] The token cache is being invalidate because project 3eeb69d949684cde978970f704b4bce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:20:46.980864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:46.983434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:46.990952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 887/1762] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:20:46 2022] DELETE /identity/v3/projects/3eeb69d949684cde978970f704b4bce2 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:46.991624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:46.997851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:47.142414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a3a47-1534-4856-83c6-77c96464b99a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:47.149265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53ab43-3900-44ee-bb2b-5a19422a1c2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:47.150510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53ab43-3900-44ee-bb2b-5a19422a1c2a service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:47.150802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53ab43-3900-44ee-bb2b-5a19422a1c2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:47.151052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53ab43-3900-44ee-bb2b-5a19422a1c2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:47.153551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834c107-69d0-41a8-8bb6-790a1daa8800 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:47.159375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1e53ab43-3900-44ee-bb2b-5a19422a1c2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:47.250303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:47.341636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53ab43-3900-44ee-bb2b-5a19422a1c2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:47.348568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53ab43-3900-44ee-bb2b-5a19422a1c2a service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:47.375013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5298d-05d8-4892-acc9-b703d5569211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:47.375220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5298d-05d8-4892-acc9-b703d5569211 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:47.375564 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5298d-05d8-4892-acc9-b703d5569211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:47.379273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5298d-05d8-4892-acc9-b703d5569211 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:47.380096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5298d-05d8-4892-acc9-b703d5569211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:47.392724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5298d-05d8-4892-acc9-b703d5569211 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:47.394724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 888/1763] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:47.399031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1ee7e-7d12-496b-8bf8-c7e8d4fb9577 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:47.402098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1ee7e-7d12-496b-8bf8-c7e8d4fb9577 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:47.402599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1ee7e-7d12-496b-8bf8-c7e8d4fb9577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:47.420273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:47.484082 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 876/1764] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:46 2022] POST /identity/v3/users => generated 301 bytes in 655 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:47.490915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a69ad-2b2e-4b1e-89fd-0d6a267e6daa None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:47.491438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a69ad-2b2e-4b1e-89fd-0d6a267e6daa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:47.492100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a69ad-2b2e-4b1e-89fd-0d6a267e6daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:47.503126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3fa1ee7e-7d12-496b-8bf8-c7e8d4fb9577 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:47.505134 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3fa1ee7e-7d12-496b-8bf8-c7e8d4fb9577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:47.519367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-342a69ad-2b2e-4b1e-89fd-0d6a267e6daa None None] MFA Rules not processed for user `15fae427f22d476fa0e378949de41292`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:47.521281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-342a69ad-2b2e-4b1e-89fd-0d6a267e6daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:47.543976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 889/1765] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:47.545038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1ee7e-7d12-496b-8bf8-c7e8d4fb9577 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:47.554877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3fa1ee7e-7d12-496b-8bf8-c7e8d4fb9577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:47.578730 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1ee7e-7d12-496b-8bf8-c7e8d4fb9577 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:47.583961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20595-7cc5-4c6e-b163-5bf9373fc6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:47.584785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20595-7cc5-4c6e-b163-5bf9373fc6ee admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:47.585088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20595-7cc5-4c6e-b163-5bf9373fc6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:47.585369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20595-7cc5-4c6e-b163-5bf9373fc6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:47.586199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac20595-7cc5-4c6e-b163-5bf9373fc6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:47.597622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac20595-7cc5-4c6e-b163-5bf9373fc6ee admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:47.631216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 890/1766] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:47.638156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20595-7cc5-4c6e-b163-5bf9373fc6ee admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:47.641320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dac20595-7cc5-4c6e-b163-5bf9373fc6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:47.656413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 877/1767] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:47.659579 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2a90a-ba55-445e-96da-b4bf29dbc3df None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:47.659957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2a90a-ba55-445e-96da-b4bf29dbc3df None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:47.660314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2a90a-ba55-445e-96da-b4bf29dbc3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:47.725963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ebf2a90a-ba55-445e-96da-b4bf29dbc3df None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:47.727414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ebf2a90a-ba55-445e-96da-b4bf29dbc3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:47.760394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 878/1768] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:47.768244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2a90a-ba55-445e-96da-b4bf29dbc3df None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:47.828404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2a90a-ba55-445e-96da-b4bf29dbc3df None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:47.845960 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e486f-2417-4b79-bd06-7614a56de2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:47.847278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e486f-2417-4b79-bd06-7614a56de2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:47.847623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e486f-2417-4b79-bd06-7614a56de2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:47.848057 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e486f-2417-4b79-bd06-7614a56de2f2 admin admin] PATH_INFO: `/v3/projects/c079038ae4c74132a55d3edce73a6c34/users/74c8cc4e18054d849380ec05fc127136/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:47.894396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e486f-2417-4b79-bd06-7614a56de2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c079038ae4c74132a55d3edce73a6c34, user_id=74c8cc4e18054d849380ec05fc127136, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:47.904167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e486f-2417-4b79-bd06-7614a56de2f2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:47.938271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 879/1769] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:47 2022] PUT /identity/v3/projects/c079038ae4c74132a55d3edce73a6c34/users/74c8cc4e18054d849380ec05fc127136/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:47.940171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e486f-2417-4b79-bd06-7614a56de2f2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:48.194604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e486f-2417-4b79-bd06-7614a56de2f2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:48.195261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20595-7cc5-4c6e-b163-5bf9373fc6ee admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:48.235148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a012e-9a41-41e4-be18-69e830ce19aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:48.238543 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a012e-9a41-41e4-be18-69e830ce19aa admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:48.244163 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a012e-9a41-41e4-be18-69e830ce19aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:48.244703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a012e-9a41-41e4-be18-69e830ce19aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:48.246053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a012e-9a41-41e4-be18-69e830ce19aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:48.248760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a012e-9a41-41e4-be18-69e830ce19aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:48.250595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a012e-9a41-41e4-be18-69e830ce19aa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:48.290411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 880/1770] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:48.292588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a012e-9a41-41e4-be18-69e830ce19aa admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:48.335280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a012e-9a41-41e4-be18-69e830ce19aa admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:48.351431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292e819-60b4-4208-9577-e788d2c7453f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:48.352398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292e819-60b4-4208-9577-e788d2c7453f service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:48.352999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292e819-60b4-4208-9577-e788d2c7453f service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:48.353602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292e819-60b4-4208-9577-e788d2c7453f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:48.356444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d292e819-60b4-4208-9577-e788d2c7453f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:48.367963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25dd01-95c5-474e-b51a-ad5dd94d04d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:48.374231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25dd01-95c5-474e-b51a-ad5dd94d04d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:48.374522 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25dd01-95c5-474e-b51a-ad5dd94d04d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:48.374799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25dd01-95c5-474e-b51a-ad5dd94d04d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:48.378564 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25dd01-95c5-474e-b51a-ad5dd94d04d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:48.389893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25dd01-95c5-474e-b51a-ad5dd94d04d5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:48.426296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 881/1771] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:48.429549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25dd01-95c5-474e-b51a-ad5dd94d04d5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:48.461083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25dd01-95c5-474e-b51a-ad5dd94d04d5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:48.472965 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495cbb6-be9a-44d1-bd3f-3bdc67f914a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:48.473918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495cbb6-be9a-44d1-bd3f-3bdc67f914a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:48.474050 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495cbb6-be9a-44d1-bd3f-3bdc67f914a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:48.474273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495cbb6-be9a-44d1-bd3f-3bdc67f914a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:48.475250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495cbb6-be9a-44d1-bd3f-3bdc67f914a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:48.477436 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495cbb6-be9a-44d1-bd3f-3bdc67f914a0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:48.506068 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292e819-60b4-4208-9577-e788d2c7453f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:48.509804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292e819-60b4-4208-9577-e788d2c7453f service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:48.526534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:48.526726 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 891/1772] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 895 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:48.527883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292e819-60b4-4208-9577-e788d2c7453f service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:48.576542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292e819-60b4-4208-9577-e788d2c7453f service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:48.586373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 882/1773] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:48 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:48.590835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc42803-cd8a-43e3-810b-0a5ecfc2bea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:48.591826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942c7c2-a74e-433b-a32e-b2be6faaa673 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:48.592220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942c7c2-a74e-433b-a32e-b2be6faaa673 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:48.592583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942c7c2-a74e-433b-a32e-b2be6faaa673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:48.602232 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc42803-cd8a-43e3-810b-0a5ecfc2bea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:48.603052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc42803-cd8a-43e3-810b-0a5ecfc2bea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:48.603715 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc42803-cd8a-43e3-810b-0a5ecfc2bea6 admin admin] PATH_INFO: `/v3/projects/c079038ae4c74132a55d3edce73a6c34/users/74c8cc4e18054d849380ec05fc127136/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:48.613014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc42803-cd8a-43e3-810b-0a5ecfc2bea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c079038ae4c74132a55d3edce73a6c34, user_id=74c8cc4e18054d849380ec05fc127136, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:48.623583 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc42803-cd8a-43e3-810b-0a5ecfc2bea6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:48.638222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f942c7c2-a74e-433b-a32e-b2be6faaa673 None None] MFA Rules not processed for user `30de794b71b343289491531dfbe37f53`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:48.645318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f942c7c2-a74e-433b-a32e-b2be6faaa673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:48.688229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 892/1774] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:48 2022] PUT /identity/v3/projects/c079038ae4c74132a55d3edce73a6c34/users/74c8cc4e18054d849380ec05fc127136/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:48.694026 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc42803-cd8a-43e3-810b-0a5ecfc2bea6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:48.720397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:48.764354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 883/1775] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:48.764354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a49531-ae72-4f94-99e4-0984c6201d07 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:48.766060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a49531-ae72-4f94-99e4-0984c6201d07 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:48.766367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a49531-ae72-4f94-99e4-0984c6201d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:48.837438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-82a49531-ae72-4f94-99e4-0984c6201d07 None None] MFA Rules not processed for user `74c8cc4e18054d849380ec05fc127136`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:48.838764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-82a49531-ae72-4f94-99e4-0984c6201d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:48.865358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc42803-cd8a-43e3-810b-0a5ecfc2bea6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:48.875374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64c91f-dce4-426c-9cbc-92be5dc2eb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:48.876508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64c91f-dce4-426c-9cbc-92be5dc2eb68 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:48.876991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64c91f-dce4-426c-9cbc-92be5dc2eb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:48.877462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64c91f-dce4-426c-9cbc-92be5dc2eb68 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:48.881509 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64c91f-dce4-426c-9cbc-92be5dc2eb68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:48.885015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64c91f-dce4-426c-9cbc-92be5dc2eb68 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:48.924968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 893/1776] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:48 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:48.926463 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64c91f-dce4-426c-9cbc-92be5dc2eb68 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:48.946289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64c91f-dce4-426c-9cbc-92be5dc2eb68 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:48.951153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f07ae7-2638-4fde-beda-44e62b9ae19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:48.951828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f07ae7-2638-4fde-beda-44e62b9ae19d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:48.952110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f07ae7-2638-4fde-beda-44e62b9ae19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:48.952383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f07ae7-2638-4fde-beda-44e62b9ae19d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:48.953117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f07ae7-2638-4fde-beda-44e62b9ae19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:48.954758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f07ae7-2638-4fde-beda-44e62b9ae19d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:48.967342 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 894/1777] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:48.972729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f07ae7-2638-4fde-beda-44e62b9ae19d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:48.997198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f07ae7-2638-4fde-beda-44e62b9ae19d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:48.999179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 884/1778] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:49.006520 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22778dc6-1132-4874-ab35-40c7703ca718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:49.008457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22778dc6-1132-4874-ab35-40c7703ca718 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:49.009190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22778dc6-1132-4874-ab35-40c7703ca718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:49.009657 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22778dc6-1132-4874-ab35-40c7703ca718 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:49.010623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a49531-ae72-4f94-99e4-0984c6201d07 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:49.010970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22778dc6-1132-4874-ab35-40c7703ca718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:49.013562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22778dc6-1132-4874-ab35-40c7703ca718 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:49.042725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 895/1779] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:49.044358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22778dc6-1132-4874-ab35-40c7703ca718 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:49.065796 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22778dc6-1132-4874-ab35-40c7703ca718 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:49.075515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a244a1-0a46-4856-b355-61dc7cf3ff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:49.076178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a244a1-0a46-4856-b355-61dc7cf3ff58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:49.076867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a244a1-0a46-4856-b355-61dc7cf3ff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:49.076867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a244a1-0a46-4856-b355-61dc7cf3ff58 admin admin] PATH_INFO: `/v3/projects/25e44cd8cfef454eb7e29d2f7daaa917/users/3c6b2e0f707c46b8af53804616534042/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:49.100112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a244a1-0a46-4856-b355-61dc7cf3ff58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e44cd8cfef454eb7e29d2f7daaa917, user_id=3c6b2e0f707c46b8af53804616534042, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:49.108284 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a244a1-0a46-4856-b355-61dc7cf3ff58 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:49.124094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a49531-ae72-4f94-99e4-0984c6201d07 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:49.128578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6063d-5783-4117-a104-0e24adf956c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:49.129871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6063d-5783-4117-a104-0e24adf956c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:49.130576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6063d-5783-4117-a104-0e24adf956c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:49.130576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6063d-5783-4117-a104-0e24adf956c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:49.137307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 896/1780] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:49 2022] PUT /identity/v3/projects/25e44cd8cfef454eb7e29d2f7daaa917/users/3c6b2e0f707c46b8af53804616534042/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:49.148950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e157a38-c556-46d5-9de7-694c2f0607ab None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:49.149608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e157a38-c556-46d5-9de7-694c2f0607ab None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:49.150614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e157a38-c556-46d5-9de7-694c2f0607ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:49.152351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6063d-5783-4117-a104-0e24adf956c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:49.153864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6063d-5783-4117-a104-0e24adf956c3 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:49.176569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 885/1781] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:49.215396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4e157a38-c556-46d5-9de7-694c2f0607ab None None] MFA Rules not processed for user `3c6b2e0f707c46b8af53804616534042`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:49.216832 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4e157a38-c556-46d5-9de7-694c2f0607ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:49.415244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6063d-5783-4117-a104-0e24adf956c3 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:49.434529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6063d-5783-4117-a104-0e24adf956c3 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:49.435325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 897/1782] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:49.437756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e157a38-c556-46d5-9de7-694c2f0607ab None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:49.573421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:49.649217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:49.728342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bbd19-fa51-4c62-af2e-23f7d201670c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:49.738147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bbd19-fa51-4c62-af2e-23f7d201670c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:49.740352 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bbd19-fa51-4c62-af2e-23f7d201670c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:49.740816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bbd19-fa51-4c62-af2e-23f7d201670c admin admin] PATH_INFO: `/v3/users/69a6c9ca9018422bbda4ffcad74716ec` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:49.746846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e157a38-c556-46d5-9de7-694c2f0607ab None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:49.747826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bbd19-fa51-4c62-af2e-23f7d201670c admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a6c9ca9018422bbda4ffcad74716ec)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:49.756591 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bbd19-fa51-4c62-af2e-23f7d201670c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:49.799939 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da12d24-86cc-4dcd-b95f-679bf4512b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:49.803578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da12d24-86cc-4dcd-b95f-679bf4512b11 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:49.808466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da12d24-86cc-4dcd-b95f-679bf4512b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:49.808920 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da12d24-86cc-4dcd-b95f-679bf4512b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:49.811964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da12d24-86cc-4dcd-b95f-679bf4512b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:49.815714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da12d24-86cc-4dcd-b95f-679bf4512b11 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:49.941079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 898/1783] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:49 2022] POST /identity/v3/projects => generated 371 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:49.942280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da12d24-86cc-4dcd-b95f-679bf4512b11 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:49.974573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ff7bbd19-fa51-4c62-af2e-23f7d201670c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a6c9ca9018422bbda4ffcad74716ec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:49.998993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ff7bbd19-fa51-4c62-af2e-23f7d201670c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a6c9ca9018422bbda4ffcad74716ec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:50.001008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ff7bbd19-fa51-4c62-af2e-23f7d201670c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a6c9ca9018422bbda4ffcad74716ec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:50.075420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ff7bbd19-fa51-4c62-af2e-23f7d201670c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a6c9ca9018422bbda4ffcad74716ec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:50.086918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ff7bbd19-fa51-4c62-af2e-23f7d201670c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a6c9ca9018422bbda4ffcad74716ec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:50.100485 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 886/1784] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:20:49 2022] DELETE /identity/v3/users/69a6c9ca9018422bbda4ffcad74716ec => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:20:50.108898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392a4f-f590-472e-82a4-2c2a0000ecce None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:50.110634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392a4f-f590-472e-82a4-2c2a0000ecce None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:50.111220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392a4f-f590-472e-82a4-2c2a0000ecce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:50.164914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a3392a4f-f590-472e-82a4-2c2a0000ecce None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:50.168570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a3392a4f-f590-472e-82a4-2c2a0000ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:50.193928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da12d24-86cc-4dcd-b95f-679bf4512b11 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:50.337020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f2d17-5701-426b-b8a6-c8a1f4ddd79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:50.338559 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f2d17-5701-426b-b8a6-c8a1f4ddd79f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:50.338953 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f2d17-5701-426b-b8a6-c8a1f4ddd79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:50.339326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f2d17-5701-426b-b8a6-c8a1f4ddd79f admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:50.340425 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f2d17-5701-426b-b8a6-c8a1f4ddd79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:50.342943 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f2d17-5701-426b-b8a6-c8a1f4ddd79f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:50.344589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 887/1785] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:50.362002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3392a4f-f590-472e-82a4-2c2a0000ecce None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:50.364262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a3392a4f-f590-472e-82a4-2c2a0000ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:50.374645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 899/1786] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:49 2022] POST /identity/v3/users => generated 292 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:50.398381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f2d17-5701-426b-b8a6-c8a1f4ddd79f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:50.403828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-330f2d17-5701-426b-b8a6-c8a1f4ddd79f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:50.440429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f2d17-5701-426b-b8a6-c8a1f4ddd79f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:50.453264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acf7ae-f9cd-42e4-8af5-4273be2c66fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:50.454044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acf7ae-f9cd-42e4-8af5-4273be2c66fc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:50.454389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acf7ae-f9cd-42e4-8af5-4273be2c66fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:50.454564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acf7ae-f9cd-42e4-8af5-4273be2c66fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:50.455310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acf7ae-f9cd-42e4-8af5-4273be2c66fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:50.459762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acf7ae-f9cd-42e4-8af5-4273be2c66fc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:50.487246 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 900/1787] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:50.488091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acf7ae-f9cd-42e4-8af5-4273be2c66fc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:50.513132 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acf7ae-f9cd-42e4-8af5-4273be2c66fc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:50.519997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:50.523871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0ac12-23ad-45c6-94f6-9aa1cb13ee09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:50.527763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0ac12-23ad-45c6-94f6-9aa1cb13ee09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:50.528076 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0ac12-23ad-45c6-94f6-9aa1cb13ee09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:50.528359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0ac12-23ad-45c6-94f6-9aa1cb13ee09 admin admin] PATH_INFO: `/v3/projects/d374f54812904e219fdf7be7a33bfe71/users/4c6a54c70a1a489c8baf861059744830/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:50.530911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3392a4f-f590-472e-82a4-2c2a0000ecce None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:50.539076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d4a57-9538-4161-ba14-9a4959fe63ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:50.539751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d4a57-9538-4161-ba14-9a4959fe63ee service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:50.540150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d4a57-9538-4161-ba14-9a4959fe63ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:50.540523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d4a57-9538-4161-ba14-9a4959fe63ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:50.569048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0ac12-23ad-45c6-94f6-9aa1cb13ee09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d374f54812904e219fdf7be7a33bfe71, user_id=4c6a54c70a1a489c8baf861059744830, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:50.570271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d4a57-9538-4161-ba14-9a4959fe63ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:50.575954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0ac12-23ad-45c6-94f6-9aa1cb13ee09 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:50.578403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d4a57-9538-4161-ba14-9a4959fe63ee service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:50.624343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 888/1789] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:50.624986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 901/1789] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:50 2022] PUT /identity/v3/projects/d374f54812904e219fdf7be7a33bfe71/users/4c6a54c70a1a489c8baf861059744830/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:50.626713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d4a57-9538-4161-ba14-9a4959fe63ee service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:50.678371 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d4a57-9538-4161-ba14-9a4959fe63ee service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:50.904309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649297e-0efc-4bca-9628-48fb244ff554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:50.905427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649297e-0efc-4bca-9628-48fb244ff554 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:50.905738 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649297e-0efc-4bca-9628-48fb244ff554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:50.906034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649297e-0efc-4bca-9628-48fb244ff554 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:50.906774 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649297e-0efc-4bca-9628-48fb244ff554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:50.923133 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649297e-0efc-4bca-9628-48fb244ff554 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:50.984414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0ac12-23ad-45c6-94f6-9aa1cb13ee09 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:50.986343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 889/1790] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:51.023055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0ac12-23ad-45c6-94f6-9aa1cb13ee09 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:51.073906 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c847aaf-5e11-4162-8f56-6170d9aa04aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:51.075516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c847aaf-5e11-4162-8f56-6170d9aa04aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:51.075516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c847aaf-5e11-4162-8f56-6170d9aa04aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:51.075516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c847aaf-5e11-4162-8f56-6170d9aa04aa admin admin] PATH_INFO: `/v3/projects/d374f54812904e219fdf7be7a33bfe71/users/4c6a54c70a1a489c8baf861059744830/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:51.086242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c847aaf-5e11-4162-8f56-6170d9aa04aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d374f54812904e219fdf7be7a33bfe71, user_id=4c6a54c70a1a489c8baf861059744830, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:51.098883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c847aaf-5e11-4162-8f56-6170d9aa04aa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:51.158392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 902/1791] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:50 2022] PUT /identity/v3/projects/d374f54812904e219fdf7be7a33bfe71/users/4c6a54c70a1a489c8baf861059744830/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:51.164234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01ff43-6119-476a-b10d-aa984adf0798 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:51.164636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01ff43-6119-476a-b10d-aa984adf0798 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:51.164876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01ff43-6119-476a-b10d-aa984adf0798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:51.267650 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3a01ff43-6119-476a-b10d-aa984adf0798 None None] MFA Rules not processed for user `4c6a54c70a1a489c8baf861059744830`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:51.271139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3a01ff43-6119-476a-b10d-aa984adf0798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:51.566771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 890/1792] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:51 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:51.588026 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c847aaf-5e11-4162-8f56-6170d9aa04aa admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:51.590096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8c847aaf-5e11-4162-8f56-6170d9aa04aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:51.645767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:51.819705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c847aaf-5e11-4162-8f56-6170d9aa04aa admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:51.839383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fcec9-027c-480c-851b-8ccbdcb6540d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:51.839383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fcec9-027c-480c-851b-8ccbdcb6540d service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:51.840334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fcec9-027c-480c-851b-8ccbdcb6540d service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:51.840334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fcec9-027c-480c-851b-8ccbdcb6540d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:51.842049 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-868fcec9-027c-480c-851b-8ccbdcb6540d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:52.003732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fcec9-027c-480c-851b-8ccbdcb6540d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:52.005545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fcec9-027c-480c-851b-8ccbdcb6540d service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:52.027665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 903/1793] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:53.611027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01ff43-6119-476a-b10d-aa984adf0798 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:53.612612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3a01ff43-6119-476a-b10d-aa984adf0798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:53.682343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01ff43-6119-476a-b10d-aa984adf0798 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:53.687291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:53.688493 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:53.688938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:53.689370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] PATH_INFO: `/v3/projects/b1b0077769d14ca6bf6922533bf27952` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:53.690772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b0077769d14ca6bf6922533bf27952)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:53.693808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:53.712265 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:53.746700 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b0077769d14ca6bf6922533bf27952', 'request_id': 'req-138d4243-e44b-4d73-b749-1de2299d99cc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:53.752395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b0077769d14ca6bf6922533bf27952', 'request_id': 'req-138d4243-e44b-4d73-b749-1de2299d99cc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:53.759583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b0077769d14ca6bf6922533bf27952', 'request_id': 'req-138d4243-e44b-4d73-b749-1de2299d99cc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:53.773017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] The token cache is being invalidate because project b1b0077769d14ca6bf6922533bf27952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:20:53.773308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:53.774000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-138d4243-e44b-4d73-b749-1de2299d99cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:53.776827 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 891/1794] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:20:53 2022] DELETE /identity/v3/projects/b1b0077769d14ca6bf6922533bf27952 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:53.781161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fcec9-027c-480c-851b-8ccbdcb6540d service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:53.792827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fcec9-027c-480c-851b-8ccbdcb6540d service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:53.853433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:53.854651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:53.854999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:53.855235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] PATH_INFO: `/v3/users/74c8cc4e18054d849380ec05fc127136` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:53.856521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74c8cc4e18054d849380ec05fc127136)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:53.858487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:53.917935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74c8cc4e18054d849380ec05fc127136'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:53.919015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74c8cc4e18054d849380ec05fc127136'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:53.919989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74c8cc4e18054d849380ec05fc127136'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:53.932033 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74c8cc4e18054d849380ec05fc127136'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:53.936461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:53.940718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74c8cc4e18054d849380ec05fc127136'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:53.957382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 904/1795] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:20:53 2022] DELETE /identity/v3/users/74c8cc4e18054d849380ec05fc127136 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:54.108071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877fddc-fbac-49aa-b0e4-da9669cac1a0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:54.108395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877fddc-fbac-49aa-b0e4-da9669cac1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:54.108703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877fddc-fbac-49aa-b0e4-da9669cac1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:54.133804 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8877fddc-fbac-49aa-b0e4-da9669cac1a0 None None] MFA Rules not processed for user `9178e89a790d48d19b25c4c22bd0ca4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:54.135233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8877fddc-fbac-49aa-b0e4-da9669cac1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:54.205773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 892/1796] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:54.506807 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:54.510273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:54.612753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71888cf-18ba-4566-a205-118f791e1c00 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:54.619212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:54.620095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:54.620375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:54.620651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] PATH_INFO: `/v3/projects/c079038ae4c74132a55d3edce73a6c34` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:54.621944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c079038ae4c74132a55d3edce73a6c34)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:54.624547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:54.690317 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c079038ae4c74132a55d3edce73a6c34', 'request_id': 'req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:54.697378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c079038ae4c74132a55d3edce73a6c34', 'request_id': 'req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:54.705798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c079038ae4c74132a55d3edce73a6c34', 'request_id': 'req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:54.729594 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] The token cache is being invalidate because project c079038ae4c74132a55d3edce73a6c34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:20:54.729927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:54.730601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6e8a04c3-1ddd-4812-a7e3-430ff286f0fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:20:54.733019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 905/1797] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:20:54 2022] DELETE /identity/v3/projects/c079038ae4c74132a55d3edce73a6c34 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:54.738132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7e07-bb49-4617-bbd4-8f1bbc9b9bf5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:54.739004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7e07-bb49-4617-bbd4-8f1bbc9b9bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:54.739504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7e07-bb49-4617-bbd4-8f1bbc9b9bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:54.760995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-927e7e07-bb49-4617-bbd4-8f1bbc9b9bf5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:54.762712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-927e7e07-bb49-4617-bbd4-8f1bbc9b9bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:54.828821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 893/1798] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:54.836801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b2135-66fa-4419-94e7-b0cdfaeab078 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:54.837994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b2135-66fa-4419-94e7-b0cdfaeab078 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:54.837994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b2135-66fa-4419-94e7-b0cdfaeab078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:54.856280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8e8b2135-66fa-4419-94e7-b0cdfaeab078 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:54.857515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8e8b2135-66fa-4419-94e7-b0cdfaeab078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:54.865330 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 906/1799] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:20:54.870268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e7e07-bb49-4617-bbd4-8f1bbc9b9bf5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:54.879636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e7e07-bb49-4617-bbd4-8f1bbc9b9bf5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:54.884863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a001f9-0143-416d-bfe9-5f15cc04aa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:54.885527 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a001f9-0143-416d-bfe9-5f15cc04aa77 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:54.885843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a001f9-0143-416d-bfe9-5f15cc04aa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:54.886122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a001f9-0143-416d-bfe9-5f15cc04aa77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:54.886854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a001f9-0143-416d-bfe9-5f15cc04aa77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:20:54.887159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a001f9-0143-416d-bfe9-5f15cc04aa77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:54.888761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a001f9-0143-416d-bfe9-5f15cc04aa77 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:54.905073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 894/1800] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:20:54.912517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b2135-66fa-4419-94e7-b0cdfaeab078 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:54.930009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b2135-66fa-4419-94e7-b0cdfaeab078 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:54.936106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170cf90-9ed8-4890-8919-d8c3680fa6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:54.936511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cf90-9ed8-4890-8919-d8c3680fa6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:54.936780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cf90-9ed8-4890-8919-d8c3680fa6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:54.937057 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cf90-9ed8-4890-8919-d8c3680fa6c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:54.937782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170cf90-9ed8-4890-8919-d8c3680fa6c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:54.939752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170cf90-9ed8-4890-8919-d8c3680fa6c8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:54.982028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 907/1801] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:54 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:54.986234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a001f9-0143-416d-bfe9-5f15cc04aa77 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:54.996925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a001f9-0143-416d-bfe9-5f15cc04aa77 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:55.031903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:55.062530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0412-d96d-4b6a-8c0b-5933c1ed3c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:55.063213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0412-d96d-4b6a-8c0b-5933c1ed3c55 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:55.064051 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0412-d96d-4b6a-8c0b-5933c1ed3c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:55.064341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0412-d96d-4b6a-8c0b-5933c1ed3c55 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:55.065042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe0412-d96d-4b6a-8c0b-5933c1ed3c55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:55.067036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe0412-d96d-4b6a-8c0b-5933c1ed3c55 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:55.093785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 895/1802] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:54 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:55.098107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170cf90-9ed8-4890-8919-d8c3680fa6c8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:55.117011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170cf90-9ed8-4890-8919-d8c3680fa6c8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:55.123102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405849fe-a1b3-46cf-b178-9ee263d0a25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:55.124649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405849fe-a1b3-46cf-b178-9ee263d0a25e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:55.125125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405849fe-a1b3-46cf-b178-9ee263d0a25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:55.125577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405849fe-a1b3-46cf-b178-9ee263d0a25e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:55.126393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405849fe-a1b3-46cf-b178-9ee263d0a25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:55.128016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405849fe-a1b3-46cf-b178-9ee263d0a25e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:55.143126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 908/1803] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:55.144353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0412-d96d-4b6a-8c0b-5933c1ed3c55 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:55.164921 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0412-d96d-4b6a-8c0b-5933c1ed3c55 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:55.170658 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dde893-aeb1-4d8b-9e70-a7aaad4e2ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:55.171331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dde893-aeb1-4d8b-9e70-a7aaad4e2ffc admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:55.171612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dde893-aeb1-4d8b-9e70-a7aaad4e2ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:55.171882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dde893-aeb1-4d8b-9e70-a7aaad4e2ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:55.172650 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dde893-aeb1-4d8b-9e70-a7aaad4e2ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:55.174222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dde893-aeb1-4d8b-9e70-a7aaad4e2ffc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:55.187352 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 896/1804] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:55.191395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405849fe-a1b3-46cf-b178-9ee263d0a25e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:55.202393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405849fe-a1b3-46cf-b178-9ee263d0a25e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:55.207763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bf4d1-a6ca-49cc-911d-e385f7ad90c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:55.208924 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bf4d1-a6ca-49cc-911d-e385f7ad90c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:55.209300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bf4d1-a6ca-49cc-911d-e385f7ad90c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:55.214500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bf4d1-a6ca-49cc-911d-e385f7ad90c5 admin admin] PATH_INFO: `/v3/projects/6e7b56b948f649e8aa349b1ec85e3ffe/users/97d5cbeaf82f42d19fcf4fa4a4334b0b/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:55.237108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bf4d1-a6ca-49cc-911d-e385f7ad90c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7b56b948f649e8aa349b1ec85e3ffe, user_id=97d5cbeaf82f42d19fcf4fa4a4334b0b, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:55.239845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bf4d1-a6ca-49cc-911d-e385f7ad90c5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:55.255902 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 909/1805] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:55 2022] PUT /identity/v3/projects/6e7b56b948f649e8aa349b1ec85e3ffe/users/97d5cbeaf82f42d19fcf4fa4a4334b0b/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:55.267226 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d4f83-a1c5-40cc-a94e-095db60e27ee None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:55.267512 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d4f83-a1c5-40cc-a94e-095db60e27ee None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:55.267792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d4f83-a1c5-40cc-a94e-095db60e27ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:55.307552 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f12d4f83-a1c5-40cc-a94e-095db60e27ee None None] MFA Rules not processed for user `97d5cbeaf82f42d19fcf4fa4a4334b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:55.309000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f12d4f83-a1c5-40cc-a94e-095db60e27ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:55.402560 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 897/1806] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:55.405143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bf4d1-a6ca-49cc-911d-e385f7ad90c5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:55.422114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:55.484720 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bf4d1-a6ca-49cc-911d-e385f7ad90c5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:55.489985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfd7e9-08b0-4e4a-8955-f60cd22b57cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:55.490659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfd7e9-08b0-4e4a-8955-f60cd22b57cf admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:55.490924 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfd7e9-08b0-4e4a-8955-f60cd22b57cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:55.491196 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfd7e9-08b0-4e4a-8955-f60cd22b57cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:55.491911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfd7e9-08b0-4e4a-8955-f60cd22b57cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:55.495116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfd7e9-08b0-4e4a-8955-f60cd22b57cf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:55.520203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 910/1807] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:55 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:20:55.524393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d4f83-a1c5-40cc-a94e-095db60e27ee None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:55.535254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d4f83-a1c5-40cc-a94e-095db60e27ee None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:55.609607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb11165-9243-4e91-ac4d-08f7b8486261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:55.623106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb11165-9243-4e91-ac4d-08f7b8486261 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:55.624281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb11165-9243-4e91-ac4d-08f7b8486261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:55.627478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb11165-9243-4e91-ac4d-08f7b8486261 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:55.628286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb11165-9243-4e91-ac4d-08f7b8486261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:55.630446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb11165-9243-4e91-ac4d-08f7b8486261 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:55.673451 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 898/1808] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:55 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:55.675771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfd7e9-08b0-4e4a-8955-f60cd22b57cf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:55.694791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfd7e9-08b0-4e4a-8955-f60cd22b57cf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:55.701030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc0501-975d-4424-a2b7-2e4a6ed6bd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:55.702103 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc0501-975d-4424-a2b7-2e4a6ed6bd38 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:55.702504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc0501-975d-4424-a2b7-2e4a6ed6bd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:55.702833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc0501-975d-4424-a2b7-2e4a6ed6bd38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:55.703698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc0501-975d-4424-a2b7-2e4a6ed6bd38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:55.705809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc0501-975d-4424-a2b7-2e4a6ed6bd38 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:55.740579 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 911/1809] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:20:55.746189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb11165-9243-4e91-ac4d-08f7b8486261 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:55.758706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb11165-9243-4e91-ac4d-08f7b8486261 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:55.774687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed88f1d-2e49-44d5-80c7-b8a0a3fcf1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:55.775396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed88f1d-2e49-44d5-80c7-b8a0a3fcf1db admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:55.775691 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed88f1d-2e49-44d5-80c7-b8a0a3fcf1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:55.776177 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed88f1d-2e49-44d5-80c7-b8a0a3fcf1db admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:55.776926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed88f1d-2e49-44d5-80c7-b8a0a3fcf1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:55.778590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed88f1d-2e49-44d5-80c7-b8a0a3fcf1db admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:55.811519 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 899/1810] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:55.813707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc0501-975d-4424-a2b7-2e4a6ed6bd38 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:55.828770 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc0501-975d-4424-a2b7-2e4a6ed6bd38 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:55.839590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ea2f4-83ec-4d0f-a8e6-c9db8489bec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:55.840903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ea2f4-83ec-4d0f-a8e6-c9db8489bec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:55.841229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ea2f4-83ec-4d0f-a8e6-c9db8489bec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:55.841668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ea2f4-83ec-4d0f-a8e6-c9db8489bec2 admin admin] PATH_INFO: `/v3/projects/3a60364174034a0b848c0c0b9377c016/users/0fd9678bc51b4e13bc142e9f5cb6d349/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:55.862493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ea2f4-83ec-4d0f-a8e6-c9db8489bec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a60364174034a0b848c0c0b9377c016, user_id=0fd9678bc51b4e13bc142e9f5cb6d349, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:55.865090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ea2f4-83ec-4d0f-a8e6-c9db8489bec2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:55.886406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 912/1811] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:55 2022] PUT /identity/v3/projects/3a60364174034a0b848c0c0b9377c016/users/0fd9678bc51b4e13bc142e9f5cb6d349/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:55.898196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484969d-1865-4517-b38a-dfa9a313e7bb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:55.898601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484969d-1865-4517-b38a-dfa9a313e7bb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:55.898962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484969d-1865-4517-b38a-dfa9a313e7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:55.966388 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e484969d-1865-4517-b38a-dfa9a313e7bb None None] MFA Rules not processed for user `0fd9678bc51b4e13bc142e9f5cb6d349`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:55.969031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e484969d-1865-4517-b38a-dfa9a313e7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:56.007419 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:56.163764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 900/1812] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:56.169513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ea2f4-83ec-4d0f-a8e6-c9db8489bec2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:56.301887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ea2f4-83ec-4d0f-a8e6-c9db8489bec2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:56.307790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b462fc7-8e41-44e1-9cf4-b288df150e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:56.309413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b462fc7-8e41-44e1-9cf4-b288df150e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:56.309912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b462fc7-8e41-44e1-9cf4-b288df150e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:56.310291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b462fc7-8e41-44e1-9cf4-b288df150e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:56.311353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b462fc7-8e41-44e1-9cf4-b288df150e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:56.313431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b462fc7-8e41-44e1-9cf4-b288df150e5e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:56.353377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 913/1813] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:20:56 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:56.357396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484969d-1865-4517-b38a-dfa9a313e7bb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:56.380501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484969d-1865-4517-b38a-dfa9a313e7bb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:56.491196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9711c5-65f2-4bbf-939f-edd08b66a727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:56.491196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9711c5-65f2-4bbf-939f-edd08b66a727 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:56.491196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9711c5-65f2-4bbf-939f-edd08b66a727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:56.491196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9711c5-65f2-4bbf-939f-edd08b66a727 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:56.491634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9711c5-65f2-4bbf-939f-edd08b66a727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:56.491634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9711c5-65f2-4bbf-939f-edd08b66a727 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:56.538532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 901/1814] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:20:56 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:20:56.542735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b462fc7-8e41-44e1-9cf4-b288df150e5e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:56.564051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b462fc7-8e41-44e1-9cf4-b288df150e5e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:56.573662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ada10-7151-4282-ac19-7d6661ed00fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:56.574591 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ada10-7151-4282-ac19-7d6661ed00fb admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:56.575758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ada10-7151-4282-ac19-7d6661ed00fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:56.576048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ada10-7151-4282-ac19-7d6661ed00fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:56.576799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ada10-7151-4282-ac19-7d6661ed00fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:56.579776 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ada10-7151-4282-ac19-7d6661ed00fb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:56.595348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 914/1815] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:56.599132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9711c5-65f2-4bbf-939f-edd08b66a727 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:56.615314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9711c5-65f2-4bbf-939f-edd08b66a727 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:56.630739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78cf0e-ed27-4134-85dd-3e01d83a0439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:56.632713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78cf0e-ed27-4134-85dd-3e01d83a0439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:56.633780 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78cf0e-ed27-4134-85dd-3e01d83a0439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:56.634112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78cf0e-ed27-4134-85dd-3e01d83a0439 admin admin] PATH_INFO: `/v3/projects/9f38ab11b5634ebc95b6969e5b7b2668/users/d4be1ef8555b4a998d346652e1207923/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:56.659347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78cf0e-ed27-4134-85dd-3e01d83a0439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f38ab11b5634ebc95b6969e5b7b2668, user_id=d4be1ef8555b4a998d346652e1207923, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:56.665465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78cf0e-ed27-4134-85dd-3e01d83a0439 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:56.704117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 902/1816] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:56 2022] PUT /identity/v3/projects/9f38ab11b5634ebc95b6969e5b7b2668/users/d4be1ef8555b4a998d346652e1207923/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:56.709083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ada10-7151-4282-ac19-7d6661ed00fb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:56.742647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ada10-7151-4282-ac19-7d6661ed00fb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:56.843928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:56.894115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580ba98-fa69-4888-b639-4893eb9c70df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:56.895079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580ba98-fa69-4888-b639-4893eb9c70df admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:56.895516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580ba98-fa69-4888-b639-4893eb9c70df admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:56.895938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580ba98-fa69-4888-b639-4893eb9c70df admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:56.896850 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580ba98-fa69-4888-b639-4893eb9c70df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:56.898823 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580ba98-fa69-4888-b639-4893eb9c70df admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:56.914385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 915/1817] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:20:56.919063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78cf0e-ed27-4134-85dd-3e01d83a0439 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:56.937697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78cf0e-ed27-4134-85dd-3e01d83a0439 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:56.947937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de348a1f-0ab6-46e3-b9e4-c222106fcc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:56.949417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de348a1f-0ab6-46e3-b9e4-c222106fcc89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:56.954083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de348a1f-0ab6-46e3-b9e4-c222106fcc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:56.957143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de348a1f-0ab6-46e3-b9e4-c222106fcc89 admin admin] PATH_INFO: `/v3/projects/9f38ab11b5634ebc95b6969e5b7b2668/users/d4be1ef8555b4a998d346652e1207923/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:56.964186 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de348a1f-0ab6-46e3-b9e4-c222106fcc89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f38ab11b5634ebc95b6969e5b7b2668, user_id=d4be1ef8555b4a998d346652e1207923, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:56.973526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de348a1f-0ab6-46e3-b9e4-c222106fcc89 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:56.989006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 903/1818] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:20:56 2022] PUT /identity/v3/projects/9f38ab11b5634ebc95b6969e5b7b2668/users/d4be1ef8555b4a998d346652e1207923/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:56.999865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5e09a-9d5e-4fa6-9d6d-2f80c7960255 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:57.000355 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5e09a-9d5e-4fa6-9d6d-2f80c7960255 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:57.000853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5e09a-9d5e-4fa6-9d6d-2f80c7960255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:57.041401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3ee5e09a-9d5e-4fa6-9d6d-2f80c7960255 None None] MFA Rules not processed for user `d4be1ef8555b4a998d346652e1207923`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:57.042841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ee5e09a-9d5e-4fa6-9d6d-2f80c7960255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:57.170439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 916/1819] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:20:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:57.187759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de348a1f-0ab6-46e3-b9e4-c222106fcc89 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:57.189934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-de348a1f-0ab6-46e3-b9e4-c222106fcc89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:57.278496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de348a1f-0ab6-46e3-b9e4-c222106fcc89 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:57.283778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da92dc-f6a9-4496-9d79-b1c07f048fea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:57.284787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da92dc-f6a9-4496-9d79-b1c07f048fea service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:57.285075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da92dc-f6a9-4496-9d79-b1c07f048fea service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:57.286339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da92dc-f6a9-4496-9d79-b1c07f048fea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:57.318843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:57.411842 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da92dc-f6a9-4496-9d79-b1c07f048fea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:57.414867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da92dc-f6a9-4496-9d79-b1c07f048fea service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:57.434088 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 904/1820] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:58.167193 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc7341-5982-4f9b-8051-da018fc6ba40 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:58.168243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc7341-5982-4f9b-8051-da018fc6ba40 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:58.168531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc7341-5982-4f9b-8051-da018fc6ba40 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:58.169998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 917/1821] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:20:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:20:58.179785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462b65-8b55-4374-85ce-01fddb6a8641 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:58.180788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462b65-8b55-4374-85ce-01fddb6a8641 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:58.181314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462b65-8b55-4374-85ce-01fddb6a8641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:58.204027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c5462b65-8b55-4374-85ce-01fddb6a8641 None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:58.205263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c5462b65-8b55-4374-85ce-01fddb6a8641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:58.316122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 905/1822] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:20:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:20:58.332621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc7341-5982-4f9b-8051-da018fc6ba40 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:58.334718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5edc7341-5982-4f9b-8051-da018fc6ba40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:58.362470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc7341-5982-4f9b-8051-da018fc6ba40 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:58.369481 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016e2cd-d0bb-4f1c-bf5c-61f760c87ce7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:58.370506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016e2cd-d0bb-4f1c-bf5c-61f760c87ce7 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:58.371019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016e2cd-d0bb-4f1c-bf5c-61f760c87ce7 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:58.371456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016e2cd-d0bb-4f1c-bf5c-61f760c87ce7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:58.382090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016e2cd-d0bb-4f1c-bf5c-61f760c87ce7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:58.385777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016e2cd-d0bb-4f1c-bf5c-61f760c87ce7 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:58.397496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 918/1823] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:59.542684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5462b65-8b55-4374-85ce-01fddb6a8641 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:59.544566 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c5462b65-8b55-4374-85ce-01fddb6a8641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:59.561130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5462b65-8b55-4374-85ce-01fddb6a8641 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:59.566256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7695e0f-5f51-43e6-a4d1-3c7ec41e48b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:59.567671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7695e0f-5f51-43e6-a4d1-3c7ec41e48b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:59.567967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7695e0f-5f51-43e6-a4d1-3c7ec41e48b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:59.568243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7695e0f-5f51-43e6-a4d1-3c7ec41e48b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:59.571765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e7695e0f-5f51-43e6-a4d1-3c7ec41e48b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:59.586311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b246033-e923-44ed-b156-3f45dbb89987 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:59.588191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b246033-e923-44ed-b156-3f45dbb89987 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:59.588642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b246033-e923-44ed-b156-3f45dbb89987 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:59.590255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 919/1824] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:20:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:20:59.660673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440aa72-c163-406b-9779-e25fc443f624 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:59.661240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440aa72-c163-406b-9779-e25fc443f624 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:59.662119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440aa72-c163-406b-9779-e25fc443f624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:59.703356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6440aa72-c163-406b-9779-e25fc443f624 None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:20:59.704992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6440aa72-c163-406b-9779-e25fc443f624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:59.713002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7695e0f-5f51-43e6-a4d1-3c7ec41e48b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:59.715907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7695e0f-5f51-43e6-a4d1-3c7ec41e48b6 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:59.725628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 920/1825] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:20:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:20:59.739597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 906/1826] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:59 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:20:59.799285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440aa72-c163-406b-9779-e25fc443f624 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:59.828594 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440aa72-c163-406b-9779-e25fc443f624 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:59.853148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437ffa3-f957-4ccb-83da-1145f4a1912a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:59.854497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437ffa3-f957-4ccb-83da-1145f4a1912a service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:59.854785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437ffa3-f957-4ccb-83da-1145f4a1912a service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:59.855061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437ffa3-f957-4ccb-83da-1145f4a1912a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:59.868779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437ffa3-f957-4ccb-83da-1145f4a1912a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:59.870549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437ffa3-f957-4ccb-83da-1145f4a1912a service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:59.888347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 921/1827] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:20:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:00.389120 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7695e0f-5f51-43e6-a4d1-3c7ec41e48b6 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:00.401076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7695e0f-5f51-43e6-a4d1-3c7ec41e48b6 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:00.405410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a2b51-0c87-436b-ae10-319eea9a8101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:00.406394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2b51-0c87-436b-ae10-319eea9a8101 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:00.406871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2b51-0c87-436b-ae10-319eea9a8101 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:00.407332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2b51-0c87-436b-ae10-319eea9a8101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:00.501931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:00.528445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a2b51-0c87-436b-ae10-319eea9a8101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:00.530020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a2b51-0c87-436b-ae10-319eea9a8101 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:00.554055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 907/1828] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:02.085834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437ffa3-f957-4ccb-83da-1145f4a1912a service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:02.098692 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437ffa3-f957-4ccb-83da-1145f4a1912a service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:02.103128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced3d58-0384-4217-8863-1fbb2710d58f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:02.103781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced3d58-0384-4217-8863-1fbb2710d58f service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:02.104066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced3d58-0384-4217-8863-1fbb2710d58f service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:02.104345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced3d58-0384-4217-8863-1fbb2710d58f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:02.114767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced3d58-0384-4217-8863-1fbb2710d58f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:02.116314 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced3d58-0384-4217-8863-1fbb2710d58f service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:02.129895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 922/1829] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:04.206433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a2b51-0c87-436b-ae10-319eea9a8101 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:04.207907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-916a2b51-0c87-436b-ae10-319eea9a8101 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:04.291135 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a2b51-0c87-436b-ae10-319eea9a8101 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:04.296271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:04.297271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:04.297826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:04.298221 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] PATH_INFO: `/v3/users/e0960f850d38472b895c150bd8c5ce87` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:04.299601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0960f850d38472b895c150bd8c5ce87)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:04.301742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:04.388148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0960f850d38472b895c150bd8c5ce87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:04.411564 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0960f850d38472b895c150bd8c5ce87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:04.413675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0960f850d38472b895c150bd8c5ce87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:04.436839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0960f850d38472b895c150bd8c5ce87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:04.446422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0960f850d38472b895c150bd8c5ce87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:04.453647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 908/1830] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:04 2022] DELETE /identity/v3/users/e0960f850d38472b895c150bd8c5ce87 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:04.457850 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84a6f7-785f-45cc-808e-b05403506ec4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:04.458940 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84a6f7-785f-45cc-808e-b05403506ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:04.459568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84a6f7-785f-45cc-808e-b05403506ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:04.480596 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ce84a6f7-785f-45cc-808e-b05403506ec4 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:04.495971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ce84a6f7-785f-45cc-808e-b05403506ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:04.638119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 923/1831] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:04.653633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:04.656788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:04.812496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e127fb3-1c83-4a5a-9525-cef6bd20bcee admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:04.825402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb0892-95f4-41aa-9d41-4a189cbbf110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:04.826113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb0892-95f4-41aa-9d41-4a189cbbf110 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:04.826398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb0892-95f4-41aa-9d41-4a189cbbf110 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:04.826676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb0892-95f4-41aa-9d41-4a189cbbf110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:04.852395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb0892-95f4-41aa-9d41-4a189cbbf110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:04.854231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb0892-95f4-41aa-9d41-4a189cbbf110 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:04.885339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 909/1832] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:21:05.253708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84a6f7-785f-45cc-808e-b05403506ec4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:05.255845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ce84a6f7-785f-45cc-808e-b05403506ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:05.273167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84a6f7-785f-45cc-808e-b05403506ec4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:05.279842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:05.280997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:05.281621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:05.282735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] PATH_INFO: `/v3/projects/1eee3cafb6d1424b81a6c543a48fe275` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:05.285242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eee3cafb6d1424b81a6c543a48fe275)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:05.287256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:05.300139 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:05.399650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eee3cafb6d1424b81a6c543a48fe275', 'request_id': 'req-814372b7-24bf-4511-9d7f-946fc6b234af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:05.407020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eee3cafb6d1424b81a6c543a48fe275', 'request_id': 'req-814372b7-24bf-4511-9d7f-946fc6b234af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:05.418985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eee3cafb6d1424b81a6c543a48fe275', 'request_id': 'req-814372b7-24bf-4511-9d7f-946fc6b234af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:05.449948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] The token cache is being invalidate because project 1eee3cafb6d1424b81a6c543a48fe275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:05.450478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:05.451306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:05.453667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 924/1833] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:05 2022] DELETE /identity/v3/projects/1eee3cafb6d1424b81a6c543a48fe275 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:05.457378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb0892-95f4-41aa-9d41-4a189cbbf110 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:05.466294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:05.474950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb0892-95f4-41aa-9d41-4a189cbbf110 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:05.645671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ca1ca-148d-4444-96a3-9734560f2386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:05.646600 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ca1ca-148d-4444-96a3-9734560f2386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:05.646885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ca1ca-148d-4444-96a3-9734560f2386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:05.647164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ca1ca-148d-4444-96a3-9734560f2386 admin admin] PATH_INFO: `/v3/users/df137d2d7ede4e368ca524adc85ab72c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:05.648366 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ca1ca-148d-4444-96a3-9734560f2386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df137d2d7ede4e368ca524adc85ab72c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:05.651029 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ca1ca-148d-4444-96a3-9734560f2386 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:05.725832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-135ca1ca-148d-4444-96a3-9734560f2386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df137d2d7ede4e368ca524adc85ab72c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:05.738223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-135ca1ca-148d-4444-96a3-9734560f2386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df137d2d7ede4e368ca524adc85ab72c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:05.739070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-135ca1ca-148d-4444-96a3-9734560f2386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df137d2d7ede4e368ca524adc85ab72c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:05.754151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-135ca1ca-148d-4444-96a3-9734560f2386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df137d2d7ede4e368ca524adc85ab72c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:05.764709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-135ca1ca-148d-4444-96a3-9734560f2386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df137d2d7ede4e368ca524adc85ab72c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:05.771541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 910/1834] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:05 2022] DELETE /identity/v3/users/df137d2d7ede4e368ca524adc85ab72c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:06.505917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:06.511631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:06.544236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814372b7-24bf-4511-9d7f-946fc6b234af admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:06.656504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:06.658407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:06.658949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:06.659449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] PATH_INFO: `/v3/projects/16b3b731b63f439daca87e778bd0747a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:06.666859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b3b731b63f439daca87e778bd0747a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:06.669204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:06.754787 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b3b731b63f439daca87e778bd0747a', 'request_id': 'req-65642339-c95e-4c01-ace5-23272e3a4b26'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:06.765108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b3b731b63f439daca87e778bd0747a', 'request_id': 'req-65642339-c95e-4c01-ace5-23272e3a4b26'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:06.785496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b3b731b63f439daca87e778bd0747a', 'request_id': 'req-65642339-c95e-4c01-ace5-23272e3a4b26'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:06.801914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] The token cache is being invalidate because project 16b3b731b63f439daca87e778bd0747a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:06.802210 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:06.803291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-65642339-c95e-4c01-ace5-23272e3a4b26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:06.805806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 925/1835] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:06 2022] DELETE /identity/v3/projects/16b3b731b63f439daca87e778bd0747a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:06.812618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a514c-5a9e-4e92-90c6-2595a80f0fcb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:06.813113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a514c-5a9e-4e92-90c6-2595a80f0fcb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:06.813202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a514c-5a9e-4e92-90c6-2595a80f0fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:06.835472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5f5a514c-5a9e-4e92-90c6-2595a80f0fcb None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:06.837349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5f5a514c-5a9e-4e92-90c6-2595a80f0fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:06.936788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 911/1836] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:06.944586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a262e9a-4b90-48aa-9e1a-4839065b5077 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:06.944897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a262e9a-4b90-48aa-9e1a-4839065b5077 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:06.945180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a262e9a-4b90-48aa-9e1a-4839065b5077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:06.971739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7a262e9a-4b90-48aa-9e1a-4839065b5077 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:06.972955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7a262e9a-4b90-48aa-9e1a-4839065b5077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:06.981677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 926/1837] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:06.987092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a514c-5a9e-4e92-90c6-2595a80f0fcb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.000010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a514c-5a9e-4e92-90c6-2595a80f0fcb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.005187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71988b-8eaf-4869-91e3-5ba1d52bb2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.005937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71988b-8eaf-4869-91e3-5ba1d52bb2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.006286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71988b-8eaf-4869-91e3-5ba1d52bb2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.006568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71988b-8eaf-4869-91e3-5ba1d52bb2e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.007329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71988b-8eaf-4869-91e3-5ba1d52bb2e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:07.007639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71988b-8eaf-4869-91e3-5ba1d52bb2e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.009545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71988b-8eaf-4869-91e3-5ba1d52bb2e5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.040022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 912/1838] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:21:07.050502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a262e9a-4b90-48aa-9e1a-4839065b5077 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.061954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a262e9a-4b90-48aa-9e1a-4839065b5077 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.066235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9b3e7-39f9-40ca-9c42-2c8bce5143a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.066904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9b3e7-39f9-40ca-9c42-2c8bce5143a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.067667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9b3e7-39f9-40ca-9c42-2c8bce5143a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.067667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9b3e7-39f9-40ca-9c42-2c8bce5143a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.068522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9b3e7-39f9-40ca-9c42-2c8bce5143a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.070715 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9b3e7-39f9-40ca-9c42-2c8bce5143a5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.094412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:07.107537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 927/1839] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:07 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:07.108435 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71988b-8eaf-4869-91e3-5ba1d52bb2e5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.121023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71988b-8eaf-4869-91e3-5ba1d52bb2e5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.132713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:07.193544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8654c-d48e-4b36-8d42-fcc8e117cd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.194273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8654c-d48e-4b36-8d42-fcc8e117cd35 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.194559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8654c-d48e-4b36-8d42-fcc8e117cd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.194839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8654c-d48e-4b36-8d42-fcc8e117cd35 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.195615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8654c-d48e-4b36-8d42-fcc8e117cd35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.197655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8654c-d48e-4b36-8d42-fcc8e117cd35 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.218510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 913/1840] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:07 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:07.222598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9b3e7-39f9-40ca-9c42-2c8bce5143a5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.234916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9b3e7-39f9-40ca-9c42-2c8bce5143a5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.240180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0751225-b7b7-4363-9d85-b6a83b300d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.240842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0751225-b7b7-4363-9d85-b6a83b300d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.241126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0751225-b7b7-4363-9d85-b6a83b300d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.241400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0751225-b7b7-4363-9d85-b6a83b300d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.242203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0751225-b7b7-4363-9d85-b6a83b300d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.244125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0751225-b7b7-4363-9d85-b6a83b300d5e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.255272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 928/1841] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:07.259785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8654c-d48e-4b36-8d42-fcc8e117cd35 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.269789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8654c-d48e-4b36-8d42-fcc8e117cd35 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.275532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1bc55-ad78-4617-902d-cc606f00540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.275790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1bc55-ad78-4617-902d-cc606f00540d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.276077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1bc55-ad78-4617-902d-cc606f00540d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.276377 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1bc55-ad78-4617-902d-cc606f00540d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.277100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1bc55-ad78-4617-902d-cc606f00540d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.278719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1bc55-ad78-4617-902d-cc606f00540d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.289162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 914/1842] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:07.293140 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0751225-b7b7-4363-9d85-b6a83b300d5e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.303034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0751225-b7b7-4363-9d85-b6a83b300d5e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.308877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d042a38-d910-42f9-946c-73ac66a840de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.309894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d042a38-d910-42f9-946c-73ac66a840de admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.310248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d042a38-d910-42f9-946c-73ac66a840de admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.310593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d042a38-d910-42f9-946c-73ac66a840de admin admin] PATH_INFO: `/v3/projects/5fa48fadc286482c85936cb5b0d56ad9/users/a4cbf767913740ae8ee1dfa507b93367/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.326547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d042a38-d910-42f9-946c-73ac66a840de admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa48fadc286482c85936cb5b0d56ad9, user_id=a4cbf767913740ae8ee1dfa507b93367, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.329303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d042a38-d910-42f9-946c-73ac66a840de admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.343030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 929/1843] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:07 2022] PUT /identity/v3/projects/5fa48fadc286482c85936cb5b0d56ad9/users/a4cbf767913740ae8ee1dfa507b93367/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:07.354813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759187e-2904-42bc-b264-6fde9ea5422a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.355105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759187e-2904-42bc-b264-6fde9ea5422a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.355390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759187e-2904-42bc-b264-6fde9ea5422a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.382772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7759187e-2904-42bc-b264-6fde9ea5422a None None] MFA Rules not processed for user `a4cbf767913740ae8ee1dfa507b93367`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:07.384255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7759187e-2904-42bc-b264-6fde9ea5422a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:07.460597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 915/1844] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:07.465223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d042a38-d910-42f9-946c-73ac66a840de admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.527553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d042a38-d910-42f9-946c-73ac66a840de admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.532292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c24bb8-b2c3-4d0e-95ec-20bb7345db13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.532962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c24bb8-b2c3-4d0e-95ec-20bb7345db13 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.533265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c24bb8-b2c3-4d0e-95ec-20bb7345db13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.533524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c24bb8-b2c3-4d0e-95ec-20bb7345db13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.534242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c24bb8-b2c3-4d0e-95ec-20bb7345db13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.536166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c24bb8-b2c3-4d0e-95ec-20bb7345db13 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.566111 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 930/1845] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:07 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:07.569987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759187e-2904-42bc-b264-6fde9ea5422a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.580453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759187e-2904-42bc-b264-6fde9ea5422a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.640346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c7af6-277a-4229-8fa6-49be2390cb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.641036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c7af6-277a-4229-8fa6-49be2390cb76 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.641316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c7af6-277a-4229-8fa6-49be2390cb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.641588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c7af6-277a-4229-8fa6-49be2390cb76 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.642320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c7af6-277a-4229-8fa6-49be2390cb76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.644244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c7af6-277a-4229-8fa6-49be2390cb76 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.667212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 916/1846] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:07 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:07.670838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c24bb8-b2c3-4d0e-95ec-20bb7345db13 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.682032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c24bb8-b2c3-4d0e-95ec-20bb7345db13 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.687017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff663514-3dc7-47a0-ad01-e6ab0a5a252f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.687691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff663514-3dc7-47a0-ad01-e6ab0a5a252f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.687993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff663514-3dc7-47a0-ad01-e6ab0a5a252f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.688273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff663514-3dc7-47a0-ad01-e6ab0a5a252f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.689104 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff663514-3dc7-47a0-ad01-e6ab0a5a252f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.690875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff663514-3dc7-47a0-ad01-e6ab0a5a252f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.702074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 931/1847] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:07.706248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c7af6-277a-4229-8fa6-49be2390cb76 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.717627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c7af6-277a-4229-8fa6-49be2390cb76 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.722438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915a88b-f9a6-4368-aba8-f261bbfb8174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.723489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a88b-f9a6-4368-aba8-f261bbfb8174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.723773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a88b-f9a6-4368-aba8-f261bbfb8174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.724051 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a88b-f9a6-4368-aba8-f261bbfb8174 admin admin] PATH_INFO: `/v3/projects/c38cc427739e497d8606cbbf9ffb7e8f/users/c5f4ba7111ea497484ee511630d4302a/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.727782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:07.742146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915a88b-f9a6-4368-aba8-f261bbfb8174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38cc427739e497d8606cbbf9ffb7e8f, user_id=c5f4ba7111ea497484ee511630d4302a, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.745317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915a88b-f9a6-4368-aba8-f261bbfb8174 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.759379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 917/1848] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:07 2022] PUT /identity/v3/projects/c38cc427739e497d8606cbbf9ffb7e8f/users/c5f4ba7111ea497484ee511630d4302a/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:07.763469 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff663514-3dc7-47a0-ad01-e6ab0a5a252f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.773396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff663514-3dc7-47a0-ad01-e6ab0a5a252f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.835646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1345306-7d59-477e-a63d-3f49630ecc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.836888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1345306-7d59-477e-a63d-3f49630ecc6b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.837368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1345306-7d59-477e-a63d-3f49630ecc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.837649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1345306-7d59-477e-a63d-3f49630ecc6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.838401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1345306-7d59-477e-a63d-3f49630ecc6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.840009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1345306-7d59-477e-a63d-3f49630ecc6b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.850152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 932/1849] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:07.855132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915a88b-f9a6-4368-aba8-f261bbfb8174 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:07.865394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915a88b-f9a6-4368-aba8-f261bbfb8174 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:07.872201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a233298-4181-450c-88c9-be6343f17518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:07.873282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a233298-4181-450c-88c9-be6343f17518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.873572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a233298-4181-450c-88c9-be6343f17518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.873898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a233298-4181-450c-88c9-be6343f17518 admin admin] PATH_INFO: `/v3/projects/c38cc427739e497d8606cbbf9ffb7e8f/users/c5f4ba7111ea497484ee511630d4302a/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.875908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a233298-4181-450c-88c9-be6343f17518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38cc427739e497d8606cbbf9ffb7e8f, user_id=c5f4ba7111ea497484ee511630d4302a, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:07.878597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a233298-4181-450c-88c9-be6343f17518 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:07.889584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 918/1850] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:07 2022] PUT /identity/v3/projects/c38cc427739e497d8606cbbf9ffb7e8f/users/c5f4ba7111ea497484ee511630d4302a/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:07.900462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dc0c4-87d2-4c06-933e-38d6e100c3b5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:07.900747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dc0c4-87d2-4c06-933e-38d6e100c3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:07.901024 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dc0c4-87d2-4c06-933e-38d6e100c3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:07.927798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4f0dc0c4-87d2-4c06-933e-38d6e100c3b5 None None] MFA Rules not processed for user `c5f4ba7111ea497484ee511630d4302a`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:07.929007 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4f0dc0c4-87d2-4c06-933e-38d6e100c3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:07.949208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:07.998336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 933/1851] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:07 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:08.009493 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a233298-4181-450c-88c9-be6343f17518 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:08.011241 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4a233298-4181-450c-88c9-be6343f17518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:08.073017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a233298-4181-450c-88c9-be6343f17518 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:08.076703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ba8d6-5a0d-4867-ac92-8f49a7467c31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:08.077348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ba8d6-5a0d-4867-ac92-8f49a7467c31 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:08.077638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ba8d6-5a0d-4867-ac92-8f49a7467c31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:08.077974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ba8d6-5a0d-4867-ac92-8f49a7467c31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:08.087417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ba8d6-5a0d-4867-ac92-8f49a7467c31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:08.088837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ba8d6-5a0d-4867-ac92-8f49a7467c31 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:08.098370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 919/1852] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:08 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:09.613185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dc0c4-87d2-4c06-933e-38d6e100c3b5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:09.698118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dc0c4-87d2-4c06-933e-38d6e100c3b5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:09.704564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:09.705679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:09.706148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:09.706575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] PATH_INFO: `/v3/users/a4cbf767913740ae8ee1dfa507b93367` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:09.707950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4cbf767913740ae8ee1dfa507b93367)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:09.711107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:09.802840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4cbf767913740ae8ee1dfa507b93367'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:09.803388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4cbf767913740ae8ee1dfa507b93367'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:09.813916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4cbf767913740ae8ee1dfa507b93367'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:09.838315 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4cbf767913740ae8ee1dfa507b93367'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:09.853947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4cbf767913740ae8ee1dfa507b93367'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:09.873197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 934/1853] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:09 2022] DELETE /identity/v3/users/a4cbf767913740ae8ee1dfa507b93367 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:09.877808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0865e88-e7ce-412a-bf07-306d92d5f36a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:09.878291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0865e88-e7ce-412a-bf07-306d92d5f36a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:09.878391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0865e88-e7ce-412a-bf07-306d92d5f36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:09.904095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f0865e88-e7ce-412a-bf07-306d92d5f36a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:09.905636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f0865e88-e7ce-412a-bf07-306d92d5f36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:10.009303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 920/1854] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:10.017499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:10.020040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:10.110968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:10.141947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce666-9bee-4d7d-8ab8-edd3eb66d1d9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:10.148119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cb175-7ff4-4764-a15f-273711d4b736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:10.149843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cb175-7ff4-4764-a15f-273711d4b736 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:10.150818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cb175-7ff4-4764-a15f-273711d4b736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:10.151109 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cb175-7ff4-4764-a15f-273711d4b736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:10.166159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cb175-7ff4-4764-a15f-273711d4b736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:10.167768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cb175-7ff4-4764-a15f-273711d4b736 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:10.180786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 935/1855] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:10.581753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0865e88-e7ce-412a-bf07-306d92d5f36a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:10.584105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f0865e88-e7ce-412a-bf07-306d92d5f36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:10.612285 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0865e88-e7ce-412a-bf07-306d92d5f36a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:10.620823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:10.621708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:10.622072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:10.622349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] PATH_INFO: `/v3/projects/5fa48fadc286482c85936cb5b0d56ad9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:10.624260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa48fadc286482c85936cb5b0d56ad9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:10.626484 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:10.708857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa48fadc286482c85936cb5b0d56ad9', 'request_id': 'req-26145a51-124e-4140-b28b-0964edc9840a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:10.720619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa48fadc286482c85936cb5b0d56ad9', 'request_id': 'req-26145a51-124e-4140-b28b-0964edc9840a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:10.728299 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa48fadc286482c85936cb5b0d56ad9', 'request_id': 'req-26145a51-124e-4140-b28b-0964edc9840a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:10.739161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:10.747708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] The token cache is being invalidate because project 5fa48fadc286482c85936cb5b0d56ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:10.748145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:10.748954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:10.754292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 921/1856] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:10 2022] DELETE /identity/v3/projects/5fa48fadc286482c85936cb5b0d56ad9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:10.755691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cb175-7ff4-4764-a15f-273711d4b736 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:10.767868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cb175-7ff4-4764-a15f-273711d4b736 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:10.848584 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:10.849538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:10.849927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:10.850276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] PATH_INFO: `/v3/users/c5f4ba7111ea497484ee511630d4302a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:10.851540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f4ba7111ea497484ee511630d4302a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:10.853897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:10.938396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f4ba7111ea497484ee511630d4302a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:10.939155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f4ba7111ea497484ee511630d4302a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:10.940101 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f4ba7111ea497484ee511630d4302a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:10.955960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f4ba7111ea497484ee511630d4302a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:10.965930 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f4ba7111ea497484ee511630d4302a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:10.994167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 936/1857] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:10 2022] DELETE /identity/v3/users/c5f4ba7111ea497484ee511630d4302a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:11.157347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:11.159326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:11.173746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26145a51-124e-4140-b28b-0964edc9840a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:11.254543 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2cae8-aee1-4d30-9b52-9eb75dd5b135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:11.255617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2cae8-aee1-4d30-9b52-9eb75dd5b135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.255951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2cae8-aee1-4d30-9b52-9eb75dd5b135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.256233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2cae8-aee1-4d30-9b52-9eb75dd5b135 admin admin] PATH_INFO: `/v3/users/97d5cbeaf82f42d19fcf4fa4a4334b0b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:11.257424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2cae8-aee1-4d30-9b52-9eb75dd5b135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97d5cbeaf82f42d19fcf4fa4a4334b0b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:11.260578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2cae8-aee1-4d30-9b52-9eb75dd5b135 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:11.318331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b7c2cae8-aee1-4d30-9b52-9eb75dd5b135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97d5cbeaf82f42d19fcf4fa4a4334b0b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:11.326969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b7c2cae8-aee1-4d30-9b52-9eb75dd5b135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97d5cbeaf82f42d19fcf4fa4a4334b0b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:11.327624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b7c2cae8-aee1-4d30-9b52-9eb75dd5b135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97d5cbeaf82f42d19fcf4fa4a4334b0b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:11.347466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b7c2cae8-aee1-4d30-9b52-9eb75dd5b135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97d5cbeaf82f42d19fcf4fa4a4334b0b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:11.357009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b7c2cae8-aee1-4d30-9b52-9eb75dd5b135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97d5cbeaf82f42d19fcf4fa4a4334b0b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:11.361376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:11.363469 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:11.364231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 922/1858] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:11 2022] DELETE /identity/v3/users/97d5cbeaf82f42d19fcf4fa4a4334b0b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:11.369321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89ad4a-6ed8-4b12-b8e9-a4288dcc5bb4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.369700 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89ad4a-6ed8-4b12-b8e9-a4288dcc5bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.370070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89ad4a-6ed8-4b12-b8e9-a4288dcc5bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:11.379298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0f853-3772-4304-93fe-38fa51a10fb9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:11.394587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ec89ad4a-6ed8-4b12-b8e9-a4288dcc5bb4 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:11.395813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ec89ad4a-6ed8-4b12-b8e9-a4288dcc5bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:11.447421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:11.448303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.448588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.448869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] PATH_INFO: `/v3/projects/c38cc427739e497d8606cbbf9ffb7e8f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:11.450090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c38cc427739e497d8606cbbf9ffb7e8f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:11.452010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:11.453510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:11.458308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 923/1859] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:11.469835 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89ad4a-6ed8-4b12-b8e9-a4288dcc5bb4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:11.472126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ec89ad4a-6ed8-4b12-b8e9-a4288dcc5bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:11.492128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:11.509135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c38cc427739e497d8606cbbf9ffb7e8f', 'request_id': 'req-9fef17fd-b294-4261-8a86-ec4df8bea33f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:11.515352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c38cc427739e497d8606cbbf9ffb7e8f', 'request_id': 'req-9fef17fd-b294-4261-8a86-ec4df8bea33f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:11.523127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c38cc427739e497d8606cbbf9ffb7e8f', 'request_id': 'req-9fef17fd-b294-4261-8a86-ec4df8bea33f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:11.537265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] The token cache is being invalidate because project c38cc427739e497d8606cbbf9ffb7e8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:11.537559 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:11.538498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9fef17fd-b294-4261-8a86-ec4df8bea33f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:11.538664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89ad4a-6ed8-4b12-b8e9-a4288dcc5bb4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:11.540440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 937/1860] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:11 2022] DELETE /identity/v3/projects/c38cc427739e497d8606cbbf9ffb7e8f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:11.544802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ce23d-e575-45bd-aa6e-ee110e1b122a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.545099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ce23d-e575-45bd-aa6e-ee110e1b122a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.545381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ce23d-e575-45bd-aa6e-ee110e1b122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:11.564010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c96ce23d-e575-45bd-aa6e-ee110e1b122a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:11.565225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c96ce23d-e575-45bd-aa6e-ee110e1b122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:11.588600 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633726e5-6c01-4305-a18c-12d975adaff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:11.589179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633726e5-6c01-4305-a18c-12d975adaff7 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.589544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633726e5-6c01-4305-a18c-12d975adaff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.589867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633726e5-6c01-4305-a18c-12d975adaff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:11.591686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-633726e5-6c01-4305-a18c-12d975adaff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:11.625068 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 938/1861] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:11.632793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8227d-1a23-4e9f-b5d4-c2efeab6b2be None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.633673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8227d-1a23-4e9f-b5d4-c2efeab6b2be None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.633983 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8227d-1a23-4e9f-b5d4-c2efeab6b2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:11.653150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8ca8227d-1a23-4e9f-b5d4-c2efeab6b2be None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:11.654404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8ca8227d-1a23-4e9f-b5d4-c2efeab6b2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:11.654755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633726e5-6c01-4305-a18c-12d975adaff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:11.656204 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633726e5-6c01-4305-a18c-12d975adaff7 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:11.662568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 939/1862] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:11.666652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 924/1863] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:21:11.666819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8227d-1a23-4e9f-b5d4-c2efeab6b2be None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:11.675838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8227d-1a23-4e9f-b5d4-c2efeab6b2be None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:11.683779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b0ee1-95c3-48f7-ab25-96d29274ca89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:11.684484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b0ee1-95c3-48f7-ab25-96d29274ca89 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.684768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b0ee1-95c3-48f7-ab25-96d29274ca89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.685441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b0ee1-95c3-48f7-ab25-96d29274ca89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:11.685804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b0ee1-95c3-48f7-ab25-96d29274ca89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:11.686114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b0ee1-95c3-48f7-ab25-96d29274ca89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:11.687772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b0ee1-95c3-48f7-ab25-96d29274ca89 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:11.701394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 940/1864] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:21:11.705686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633726e5-6c01-4305-a18c-12d975adaff7 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:11.717773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633726e5-6c01-4305-a18c-12d975adaff7 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:11.724170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59339616-2a2d-4c86-8fee-afd5b01d974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:11.725089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59339616-2a2d-4c86-8fee-afd5b01d974f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.725447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59339616-2a2d-4c86-8fee-afd5b01d974f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.725834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59339616-2a2d-4c86-8fee-afd5b01d974f admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:11.727431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59339616-2a2d-4c86-8fee-afd5b01d974f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:11.730432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59339616-2a2d-4c86-8fee-afd5b01d974f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:11.761803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 925/1865] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:11 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:11.763502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b0ee1-95c3-48f7-ab25-96d29274ca89 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:11.775387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b0ee1-95c3-48f7-ab25-96d29274ca89 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:11.844648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fb87c-0c72-4089-8692-93115c840d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:11.845263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fb87c-0c72-4089-8692-93115c840d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.845974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fb87c-0c72-4089-8692-93115c840d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.845974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fb87c-0c72-4089-8692-93115c840d34 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:11.846578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fb87c-0c72-4089-8692-93115c840d34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:11.848876 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fb87c-0c72-4089-8692-93115c840d34 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:11.870908 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:11.877435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 941/1866] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:11 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:11.881944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59339616-2a2d-4c86-8fee-afd5b01d974f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:11.898171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59339616-2a2d-4c86-8fee-afd5b01d974f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:11.904139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6f599-6930-42d3-8f2f-2ff544aff368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:11.904934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6f599-6930-42d3-8f2f-2ff544aff368 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.905294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6f599-6930-42d3-8f2f-2ff544aff368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.905648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6f599-6930-42d3-8f2f-2ff544aff368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:11.906636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6f599-6930-42d3-8f2f-2ff544aff368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:11.908644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6f599-6930-42d3-8f2f-2ff544aff368 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:11.923359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 926/1867] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:11.927341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fb87c-0c72-4089-8692-93115c840d34 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:11.947734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fb87c-0c72-4089-8692-93115c840d34 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:11.954468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50a067-d94b-4d12-8ce7-d2eeb1149315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:11.955454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50a067-d94b-4d12-8ce7-d2eeb1149315 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.955854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50a067-d94b-4d12-8ce7-d2eeb1149315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.956247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50a067-d94b-4d12-8ce7-d2eeb1149315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:11.957482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50a067-d94b-4d12-8ce7-d2eeb1149315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:11.958913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50a067-d94b-4d12-8ce7-d2eeb1149315 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:11.974893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 942/1868] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:11.976111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6f599-6930-42d3-8f2f-2ff544aff368 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:11.987813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6f599-6930-42d3-8f2f-2ff544aff368 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:11.993817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086dae7f-6156-4bad-815d-ba970ff8b86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:11.994813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dae7f-6156-4bad-815d-ba970ff8b86a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:11.995177 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dae7f-6156-4bad-815d-ba970ff8b86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:11.995694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dae7f-6156-4bad-815d-ba970ff8b86a admin admin] PATH_INFO: `/v3/projects/ea555c7ed5f5422eaaf502d18dfc0af3/users/de3dbe3506a841d9aac3d708cc380e76/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:12.012633 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086dae7f-6156-4bad-815d-ba970ff8b86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea555c7ed5f5422eaaf502d18dfc0af3, user_id=de3dbe3506a841d9aac3d708cc380e76, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:12.016077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086dae7f-6156-4bad-815d-ba970ff8b86a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:12.041781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 927/1869] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:11 2022] PUT /identity/v3/projects/ea555c7ed5f5422eaaf502d18dfc0af3/users/de3dbe3506a841d9aac3d708cc380e76/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:12.048520 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf70064-b462-4d19-9159-6bc16f370ca5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:12.049063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf70064-b462-4d19-9159-6bc16f370ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:12.049164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf70064-b462-4d19-9159-6bc16f370ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:12.061063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086dae7f-6156-4bad-815d-ba970ff8b86a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:12.065509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-086dae7f-6156-4bad-815d-ba970ff8b86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:12.108956 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1cf70064-b462-4d19-9159-6bc16f370ca5 None None] MFA Rules not processed for user `de3dbe3506a841d9aac3d708cc380e76`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:12.110240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1cf70064-b462-4d19-9159-6bc16f370ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:12.146097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:12.224802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 943/1870] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:12.228617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086dae7f-6156-4bad-815d-ba970ff8b86a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:12.230664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf70064-b462-4d19-9159-6bc16f370ca5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:12.234930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:12.235816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:12.236191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:12.236467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] PATH_INFO: `/v3/projects/6e7b56b948f649e8aa349b1ec85e3ffe` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:12.237793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e7b56b948f649e8aa349b1ec85e3ffe)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:12.240053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:12.245812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf70064-b462-4d19-9159-6bc16f370ca5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:12.251759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95425365-050c-49b5-82fa-b16b04101fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:12.252680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425365-050c-49b5-82fa-b16b04101fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:12.253032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425365-050c-49b5-82fa-b16b04101fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:12.253414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425365-050c-49b5-82fa-b16b04101fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:12.254262 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95425365-050c-49b5-82fa-b16b04101fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:12.256911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95425365-050c-49b5-82fa-b16b04101fd4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:12.309347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 944/1871] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:12 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:12.310630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e7b56b948f649e8aa349b1ec85e3ffe', 'request_id': 'req-b0603ef2-e503-4373-ab2a-158f98661e7f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:12.313147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95425365-050c-49b5-82fa-b16b04101fd4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:12.328616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e7b56b948f649e8aa349b1ec85e3ffe', 'request_id': 'req-b0603ef2-e503-4373-ab2a-158f98661e7f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:12.338355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e7b56b948f649e8aa349b1ec85e3ffe', 'request_id': 'req-b0603ef2-e503-4373-ab2a-158f98661e7f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:12.361771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] The token cache is being invalidate because project 6e7b56b948f649e8aa349b1ec85e3ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:12.362126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:12.363024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:12.366584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 928/1872] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:12 2022] DELETE /identity/v3/projects/6e7b56b948f649e8aa349b1ec85e3ffe => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:12.372751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:12.375197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:12.436383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95425365-050c-49b5-82fa-b16b04101fd4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:12.441605 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bea4a-fa17-4a3d-b3a7-8264c8db0627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:12.442678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bea4a-fa17-4a3d-b3a7-8264c8db0627 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:12.442971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bea4a-fa17-4a3d-b3a7-8264c8db0627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:12.443254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bea4a-fa17-4a3d-b3a7-8264c8db0627 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:12.443955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bea4a-fa17-4a3d-b3a7-8264c8db0627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:12.447532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bea4a-fa17-4a3d-b3a7-8264c8db0627 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:12.493225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0603ef2-e503-4373-ab2a-158f98661e7f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:12.501287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:12.502671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:12.503032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:12.503493 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] PATH_INFO: `/v3/users/0fd9678bc51b4e13bc142e9f5cb6d349` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:12.505528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fd9678bc51b4e13bc142e9f5cb6d349)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:12.511918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:12.521834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 945/1873] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:12 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:12.523509 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bea4a-fa17-4a3d-b3a7-8264c8db0627 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:12.528897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b56bea4a-fa17-4a3d-b3a7-8264c8db0627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:12.553427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bea4a-fa17-4a3d-b3a7-8264c8db0627 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:12.562075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b5d1b-a2c0-4d58-b706-37880e07370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:12.563120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5d1b-a2c0-4d58-b706-37880e07370d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:12.563645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5d1b-a2c0-4d58-b706-37880e07370d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:12.564132 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5d1b-a2c0-4d58-b706-37880e07370d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:12.565075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b5d1b-a2c0-4d58-b706-37880e07370d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:12.567195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b5d1b-a2c0-4d58-b706-37880e07370d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:12.580319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 946/1874] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:21:12.584555 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b5d1b-a2c0-4d58-b706-37880e07370d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:12.588972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fd9678bc51b4e13bc142e9f5cb6d349'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:12.599800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b5d1b-a2c0-4d58-b706-37880e07370d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:12.607016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795fc2f-b0b1-470e-aaae-44dc594f7791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:12.608153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fd9678bc51b4e13bc142e9f5cb6d349'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:12.608605 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795fc2f-b0b1-470e-aaae-44dc594f7791 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:12.609239 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fd9678bc51b4e13bc142e9f5cb6d349'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:12.609378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795fc2f-b0b1-470e-aaae-44dc594f7791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:12.609989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795fc2f-b0b1-470e-aaae-44dc594f7791 admin admin] PATH_INFO: `/v3/projects/fff1429fa6e54e65abb5907364fa5f8e/users/adacb45cd7b14e84b581da79d7620281/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:12.621234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:12.630109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fd9678bc51b4e13bc142e9f5cb6d349'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:12.640759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795fc2f-b0b1-470e-aaae-44dc594f7791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff1429fa6e54e65abb5907364fa5f8e, user_id=adacb45cd7b14e84b581da79d7620281, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:12.640957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fd9678bc51b4e13bc142e9f5cb6d349'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:12.643502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795fc2f-b0b1-470e-aaae-44dc594f7791 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:12.647929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 929/1875] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:12 2022] DELETE /identity/v3/users/0fd9678bc51b4e13bc142e9f5cb6d349 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:12.662875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 947/1876] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:12 2022] PUT /identity/v3/projects/fff1429fa6e54e65abb5907364fa5f8e/users/adacb45cd7b14e84b581da79d7620281/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:12.667183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:12.668791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:12.749123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd6600-832a-4f1b-94ea-1cd55ec974f0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:12.829443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b97a9b-d788-4d57-82b1-0494da54c241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:12.830173 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b97a9b-d788-4d57-82b1-0494da54c241 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:12.830463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b97a9b-d788-4d57-82b1-0494da54c241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:12.830760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b97a9b-d788-4d57-82b1-0494da54c241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:12.831477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b97a9b-d788-4d57-82b1-0494da54c241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:12.833129 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b97a9b-d788-4d57-82b1-0494da54c241 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:12.848195 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 930/1877] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:12.849475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795fc2f-b0b1-470e-aaae-44dc594f7791 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:12.860642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795fc2f-b0b1-470e-aaae-44dc594f7791 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:12.865400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d989ca-0a01-4518-b938-6c485bc78ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:12.866274 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d989ca-0a01-4518-b938-6c485bc78ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:12.866555 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d989ca-0a01-4518-b938-6c485bc78ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:12.866830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d989ca-0a01-4518-b938-6c485bc78ac2 admin admin] PATH_INFO: `/v3/projects/fff1429fa6e54e65abb5907364fa5f8e/users/adacb45cd7b14e84b581da79d7620281/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:12.869695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d989ca-0a01-4518-b938-6c485bc78ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff1429fa6e54e65abb5907364fa5f8e, user_id=adacb45cd7b14e84b581da79d7620281, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:12.872386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d989ca-0a01-4518-b938-6c485bc78ac2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:12.878424 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:12.885430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 948/1878] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:12 2022] PUT /identity/v3/projects/fff1429fa6e54e65abb5907364fa5f8e/users/adacb45cd7b14e84b581da79d7620281/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:12.895629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438bdf5-1247-4eca-91be-bae414767022 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:12.895932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438bdf5-1247-4eca-91be-bae414767022 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:12.896219 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438bdf5-1247-4eca-91be-bae414767022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:12.922574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3438bdf5-1247-4eca-91be-bae414767022 None None] MFA Rules not processed for user `adacb45cd7b14e84b581da79d7620281`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:12.923795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3438bdf5-1247-4eca-91be-bae414767022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:12.947829 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d989ca-0a01-4518-b938-6c485bc78ac2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:12.949910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-84d989ca-0a01-4518-b938-6c485bc78ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:13.003214 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 931/1879] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:13.017852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438bdf5-1247-4eca-91be-bae414767022 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:13.020115 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3438bdf5-1247-4eca-91be-bae414767022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:13.027310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:13.030078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d989ca-0a01-4518-b938-6c485bc78ac2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:13.035036 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:13.036076 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:13.036209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:13.036442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] PATH_INFO: `/v3/projects/3a60364174034a0b848c0c0b9377c016` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:13.038064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a60364174034a0b848c0c0b9377c016)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:13.040965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:13.092458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a60364174034a0b848c0c0b9377c016', 'request_id': 'req-148b320e-8d0f-4345-8cf1-c9c62f7d8865'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.095320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438bdf5-1247-4eca-91be-bae414767022 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:13.103000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a60364174034a0b848c0c0b9377c016', 'request_id': 'req-148b320e-8d0f-4345-8cf1-c9c62f7d8865'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.111283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a60364174034a0b848c0c0b9377c016', 'request_id': 'req-148b320e-8d0f-4345-8cf1-c9c62f7d8865'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.123724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] The token cache is being invalidate because project 3a60364174034a0b848c0c0b9377c016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:13.124020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.124689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.126320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 949/1880] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:12 2022] DELETE /identity/v3/projects/3a60364174034a0b848c0c0b9377c016 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:13.130695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:13.132581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:13.154603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf0139-ae7d-4318-b3f4-fe773f8933c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:13.155262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf0139-ae7d-4318-b3f4-fe773f8933c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:13.155543 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf0139-ae7d-4318-b3f4-fe773f8933c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:13.155823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf0139-ae7d-4318-b3f4-fe773f8933c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:13.157626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-45bf0139-ae7d-4318-b3f4-fe773f8933c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:13.199028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b320e-8d0f-4345-8cf1-c9c62f7d8865 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:13.203798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:13.205055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:13.205336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:13.205949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] PATH_INFO: `/v3/users/d4be1ef8555b4a998d346652e1207923` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:13.207136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4be1ef8555b4a998d346652e1207923)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:13.209124 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:13.226056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf0139-ae7d-4318-b3f4-fe773f8933c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:13.228291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf0139-ae7d-4318-b3f4-fe773f8933c6 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:13.241773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 932/1881] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:21:13.245459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:13.262788 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4be1ef8555b4a998d346652e1207923'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.263606 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4be1ef8555b4a998d346652e1207923'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.264277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4be1ef8555b4a998d346652e1207923'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.277139 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4be1ef8555b4a998d346652e1207923'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.284863 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4be1ef8555b4a998d346652e1207923'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.302212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 950/1882] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:13 2022] DELETE /identity/v3/users/d4be1ef8555b4a998d346652e1207923 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:13.796293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf0139-ae7d-4318-b3f4-fe773f8933c6 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:13.808714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf0139-ae7d-4318-b3f4-fe773f8933c6 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:13.880438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:13.881257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:13.881521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:13.881824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] PATH_INFO: `/v3/projects/9f38ab11b5634ebc95b6969e5b7b2668` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:13.883765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f38ab11b5634ebc95b6969e5b7b2668)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:13.885881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:13.897907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:13.905557 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:13.907621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:13.922042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22d710-548d-43b6-8553-55094f53f08d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:13.926974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2109d-f650-4900-9722-13f831c41aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:13.927848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2109d-f650-4900-9722-13f831c41aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:13.928132 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2109d-f650-4900-9722-13f831c41aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:13.928407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2109d-f650-4900-9722-13f831c41aa8 admin admin] PATH_INFO: `/v3/users/de3dbe3506a841d9aac3d708cc380e76` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:13.930184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2109d-f650-4900-9722-13f831c41aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de3dbe3506a841d9aac3d708cc380e76)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:13.932372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2109d-f650-4900-9722-13f831c41aa8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:13.938714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f38ab11b5634ebc95b6969e5b7b2668', 'request_id': 'req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.944260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f38ab11b5634ebc95b6969e5b7b2668', 'request_id': 'req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.951581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f38ab11b5634ebc95b6969e5b7b2668', 'request_id': 'req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.966754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] The token cache is being invalidate because project 9f38ab11b5634ebc95b6969e5b7b2668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:13.967038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.967682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a38d7a55-2cc6-48d3-a9cb-69a675f9e316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.969366 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 933/1883] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:13 2022] DELETE /identity/v3/projects/9f38ab11b5634ebc95b6969e5b7b2668 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:13.974027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf577eaf-f17a-4a4c-8ce1-33f02384e233 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:13.974317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf577eaf-f17a-4a4c-8ce1-33f02384e233 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:13.974593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf577eaf-f17a-4a4c-8ce1-33f02384e233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:13.982827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ebb2109d-f650-4900-9722-13f831c41aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de3dbe3506a841d9aac3d708cc380e76'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.983488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ebb2109d-f650-4900-9722-13f831c41aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de3dbe3506a841d9aac3d708cc380e76'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.984125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ebb2109d-f650-4900-9722-13f831c41aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de3dbe3506a841d9aac3d708cc380e76'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:13.992576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cf577eaf-f17a-4a4c-8ce1-33f02384e233 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:13.993974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf577eaf-f17a-4a4c-8ce1-33f02384e233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:13.996917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ebb2109d-f650-4900-9722-13f831c41aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de3dbe3506a841d9aac3d708cc380e76'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.004254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ebb2109d-f650-4900-9722-13f831c41aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de3dbe3506a841d9aac3d708cc380e76'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.018513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 951/1884] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:13 2022] DELETE /identity/v3/users/de3dbe3506a841d9aac3d708cc380e76 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:14.022861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a1c8c-9d3c-43ec-9ec1-c2c7b0b9c7d1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.023155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a1c8c-9d3c-43ec-9ec1-c2c7b0b9c7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.023413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a1c8c-9d3c-43ec-9ec1-c2c7b0b9c7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.042290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c21a1c8c-9d3c-43ec-9ec1-c2c7b0b9c7d1 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:14.043542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c21a1c8c-9d3c-43ec-9ec1-c2c7b0b9c7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:14.060365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 934/1885] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:14.069012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f4e02-8d8c-4461-81d7-05ccb50b37d1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.069309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f4e02-8d8c-4461-81d7-05ccb50b37d1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.069598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f4e02-8d8c-4461-81d7-05ccb50b37d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.089108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ea7f4e02-8d8c-4461-81d7-05ccb50b37d1 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:14.090351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ea7f4e02-8d8c-4461-81d7-05ccb50b37d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:14.103832 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 952/1886] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:14.113316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a1c8c-9d3c-43ec-9ec1-c2c7b0b9c7d1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.114713 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c21a1c8c-9d3c-43ec-9ec1-c2c7b0b9c7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:14.138039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:14.161976 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 935/1887] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:14.166874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f4e02-8d8c-4461-81d7-05ccb50b37d1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.176044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f4e02-8d8c-4461-81d7-05ccb50b37d1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.178714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a1c8c-9d3c-43ec-9ec1-c2c7b0b9c7d1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.181610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd3d1-8a89-40a4-be37-8b5baa7a5af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.182315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd3d1-8a89-40a4-be37-8b5baa7a5af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.182595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd3d1-8a89-40a4-be37-8b5baa7a5af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.182873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd3d1-8a89-40a4-be37-8b5baa7a5af8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.183241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fdc57-e311-48a5-8a1c-3462818a5ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.183596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bd3d1-8a89-40a4-be37-8b5baa7a5af8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:14.183896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fdc57-e311-48a5-8a1c-3462818a5ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.183994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bd3d1-8a89-40a4-be37-8b5baa7a5af8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.184185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fdc57-e311-48a5-8a1c-3462818a5ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.184465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fdc57-e311-48a5-8a1c-3462818a5ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.185487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bd3d1-8a89-40a4-be37-8b5baa7a5af8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.194269 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fdc57-e311-48a5-8a1c-3462818a5ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.195865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fdc57-e311-48a5-8a1c-3462818a5ca2 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.197919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 936/1888] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:21:14.202175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd3d1-8a89-40a4-be37-8b5baa7a5af8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.206698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 953/1889] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:21:14.212151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd3d1-8a89-40a4-be37-8b5baa7a5af8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.217215 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3bea1-2cb7-415b-ba95-c3679311e762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.217916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3bea1-2cb7-415b-ba95-c3679311e762 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.218208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3bea1-2cb7-415b-ba95-c3679311e762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.218488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3bea1-2cb7-415b-ba95-c3679311e762 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.219182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3bea1-2cb7-415b-ba95-c3679311e762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.221106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3bea1-2cb7-415b-ba95-c3679311e762 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.259839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 937/1890] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:14 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:14.263600 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fdc57-e311-48a5-8a1c-3462818a5ca2 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.275831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fdc57-e311-48a5-8a1c-3462818a5ca2 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.339500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c067181-c71b-4578-9757-bd8d1c93ae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.340414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c067181-c71b-4578-9757-bd8d1c93ae0c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.340706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c067181-c71b-4578-9757-bd8d1c93ae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.340997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c067181-c71b-4578-9757-bd8d1c93ae0c admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.341776 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c067181-c71b-4578-9757-bd8d1c93ae0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.343869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c067181-c71b-4578-9757-bd8d1c93ae0c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.375783 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 954/1891] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:14 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:14.379496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3bea1-2cb7-415b-ba95-c3679311e762 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.391203 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3bea1-2cb7-415b-ba95-c3679311e762 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.396752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeadc24-1cb6-4796-97ae-c7697d83645b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.397432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeadc24-1cb6-4796-97ae-c7697d83645b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.397742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeadc24-1cb6-4796-97ae-c7697d83645b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.398032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeadc24-1cb6-4796-97ae-c7697d83645b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.399288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeadc24-1cb6-4796-97ae-c7697d83645b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.401034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeadc24-1cb6-4796-97ae-c7697d83645b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.402773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:14.416624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 938/1892] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:14.419503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c067181-c71b-4578-9757-bd8d1c93ae0c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.430911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c067181-c71b-4578-9757-bd8d1c93ae0c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.438502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcba22-8c18-44e2-8159-05c1d9134f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.439193 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcba22-8c18-44e2-8159-05c1d9134f42 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.439510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcba22-8c18-44e2-8159-05c1d9134f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.439816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcba22-8c18-44e2-8159-05c1d9134f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.440718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcba22-8c18-44e2-8159-05c1d9134f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.442488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcba22-8c18-44e2-8159-05c1d9134f42 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.453085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 955/1893] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:14.457336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeadc24-1cb6-4796-97ae-c7697d83645b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.468179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeadc24-1cb6-4796-97ae-c7697d83645b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.472968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533477d4-d1e3-4629-a207-28e59f9274aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.473887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533477d4-d1e3-4629-a207-28e59f9274aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.474176 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533477d4-d1e3-4629-a207-28e59f9274aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.474454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533477d4-d1e3-4629-a207-28e59f9274aa admin admin] PATH_INFO: `/v3/projects/60e88317d3164c3b9406608ca9414ce4/users/c1c0ea9425954868a239b259b11890de/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.489326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533477d4-d1e3-4629-a207-28e59f9274aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=60e88317d3164c3b9406608ca9414ce4, user_id=c1c0ea9425954868a239b259b11890de, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.492692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533477d4-d1e3-4629-a207-28e59f9274aa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.506173 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 939/1894] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:14 2022] PUT /identity/v3/projects/60e88317d3164c3b9406608ca9414ce4/users/c1c0ea9425954868a239b259b11890de/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:14.514858 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcba22-8c18-44e2-8159-05c1d9134f42 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.516743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cbbcba22-8c18-44e2-8159-05c1d9134f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:14.517558 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ffda-9681-4165-9450-e2d6d4546405 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.517928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ffda-9681-4165-9450-e2d6d4546405 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.518213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ffda-9681-4165-9450-e2d6d4546405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.529921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcba22-8c18-44e2-8159-05c1d9134f42 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.551096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a130ffda-9681-4165-9450-e2d6d4546405 None None] MFA Rules not processed for user `c1c0ea9425954868a239b259b11890de`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:14.552539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a130ffda-9681-4165-9450-e2d6d4546405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:14.599735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.600612 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.600891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.601170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] PATH_INFO: `/v3/projects/ea555c7ed5f5422eaaf502d18dfc0af3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.602558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea555c7ed5f5422eaaf502d18dfc0af3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.604506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.623519 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 940/1895] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:14.627904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130ffda-9681-4165-9450-e2d6d4546405 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.639342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130ffda-9681-4165-9450-e2d6d4546405 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.650361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:14.654719 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea555c7ed5f5422eaaf502d18dfc0af3', 'request_id': 'req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.662268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea555c7ed5f5422eaaf502d18dfc0af3', 'request_id': 'req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.677191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea555c7ed5f5422eaaf502d18dfc0af3', 'request_id': 'req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.692817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] The token cache is being invalidate because project ea555c7ed5f5422eaaf502d18dfc0af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:14.693129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.694435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.699219 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 956/1896] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:14 2022] DELETE /identity/v3/projects/ea555c7ed5f5422eaaf502d18dfc0af3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:14.701403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.703493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:14.710573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489b080-a7d8-41b9-972e-39021c46e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.712112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489b080-a7d8-41b9-972e-39021c46e28c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.712404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489b080-a7d8-41b9-972e-39021c46e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.712687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489b080-a7d8-41b9-972e-39021c46e28c admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.714114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489b080-a7d8-41b9-972e-39021c46e28c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.717119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489b080-a7d8-41b9-972e-39021c46e28c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.717489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183993c-3936-4c2f-8a5d-17f1e7d0cb51 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.722369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.723262 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.723549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.723827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] PATH_INFO: `/v3/users/adacb45cd7b14e84b581da79d7620281` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.725056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adacb45cd7b14e84b581da79d7620281)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.729225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.757325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 941/1897] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:14 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:14.764017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489b080-a7d8-41b9-972e-39021c46e28c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.766091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f489b080-a7d8-41b9-972e-39021c46e28c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:14.784349 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adacb45cd7b14e84b581da79d7620281'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.785412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adacb45cd7b14e84b581da79d7620281'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.785654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adacb45cd7b14e84b581da79d7620281'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.802395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adacb45cd7b14e84b581da79d7620281'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.810646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adacb45cd7b14e84b581da79d7620281'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:14.826629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:14.833385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 957/1898] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:14 2022] DELETE /identity/v3/users/adacb45cd7b14e84b581da79d7620281 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:14.849464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489b080-a7d8-41b9-972e-39021c46e28c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.930964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc7e7a-e501-4696-bc01-0eb24e8c2aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.930964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc7e7a-e501-4696-bc01-0eb24e8c2aba admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.930964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc7e7a-e501-4696-bc01-0eb24e8c2aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.930964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc7e7a-e501-4696-bc01-0eb24e8c2aba admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.931235 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc7e7a-e501-4696-bc01-0eb24e8c2aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.931235 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc7e7a-e501-4696-bc01-0eb24e8c2aba admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.953674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 942/1899] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:14 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:14.957999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:14.959697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:14.976111 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e9581-2592-4fde-9b75-619b427771b0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:14.981548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a65d-db57-4603-9550-b6bc8097cc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:14.982306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a65d-db57-4603-9550-b6bc8097cc00 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:14.982592 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a65d-db57-4603-9550-b6bc8097cc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:14.982871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a65d-db57-4603-9550-b6bc8097cc00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:14.983625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6a65d-db57-4603-9550-b6bc8097cc00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:14.985406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6a65d-db57-4603-9550-b6bc8097cc00 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:14.997197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 958/1900] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:15.001173 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc7e7a-e501-4696-bc01-0eb24e8c2aba admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.018728 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc7e7a-e501-4696-bc01-0eb24e8c2aba admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.025110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582822e2-60fc-4472-80e6-bdaa36114e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.026077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582822e2-60fc-4472-80e6-bdaa36114e62 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.026365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582822e2-60fc-4472-80e6-bdaa36114e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.026646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582822e2-60fc-4472-80e6-bdaa36114e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.027383 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582822e2-60fc-4472-80e6-bdaa36114e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.029000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582822e2-60fc-4472-80e6-bdaa36114e62 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.041197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 943/1901] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:15.045284 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a65d-db57-4603-9550-b6bc8097cc00 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.056364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a65d-db57-4603-9550-b6bc8097cc00 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.061352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00d6cf-bd23-49d2-a143-e3db68b1ff33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.062404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00d6cf-bd23-49d2-a143-e3db68b1ff33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.062762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00d6cf-bd23-49d2-a143-e3db68b1ff33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.063047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00d6cf-bd23-49d2-a143-e3db68b1ff33 admin admin] PATH_INFO: `/v3/projects/8de4f875595a4f7185770ed29dc00e00/users/8267155806c840ea8590fc00e37cf4f4/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.082862 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00d6cf-bd23-49d2-a143-e3db68b1ff33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8de4f875595a4f7185770ed29dc00e00, user_id=8267155806c840ea8590fc00e37cf4f4, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.085651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00d6cf-bd23-49d2-a143-e3db68b1ff33 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.105802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 959/1902] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:15 2022] PUT /identity/v3/projects/8de4f875595a4f7185770ed29dc00e00/users/8267155806c840ea8590fc00e37cf4f4/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:15.114937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12e0e8-4fa1-45a0-ae44-1c981b0bb44b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.114937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12e0e8-4fa1-45a0-ae44-1c981b0bb44b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.115158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12e0e8-4fa1-45a0-ae44-1c981b0bb44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.145106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ab12e0e8-4fa1-45a0-ae44-1c981b0bb44b None None] MFA Rules not processed for user `8267155806c840ea8590fc00e37cf4f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:15.146437 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ab12e0e8-4fa1-45a0-ae44-1c981b0bb44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:15.154237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00d6cf-bd23-49d2-a143-e3db68b1ff33 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.155970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7a00d6cf-bd23-49d2-a143-e3db68b1ff33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:15.220007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 944/1903] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:15.224410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12e0e8-4fa1-45a0-ae44-1c981b0bb44b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.224667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00d6cf-bd23-49d2-a143-e3db68b1ff33 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.229639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.230743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.230969 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.231666 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] PATH_INFO: `/v3/projects/fff1429fa6e54e65abb5907364fa5f8e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.233099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff1429fa6e54e65abb5907364fa5f8e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.235388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.236156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12e0e8-4fa1-45a0-ae44-1c981b0bb44b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.240978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cf482-80d9-4233-bb40-ac6b824e5853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.241645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cf482-80d9-4233-bb40-ac6b824e5853 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.241963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cf482-80d9-4233-bb40-ac6b824e5853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.242870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cf482-80d9-4233-bb40-ac6b824e5853 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.243576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1cf482-80d9-4233-bb40-ac6b824e5853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.245481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1cf482-80d9-4233-bb40-ac6b824e5853 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.247948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:15.274152 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 945/1904] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:15 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:15.278224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cf482-80d9-4233-bb40-ac6b824e5853 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.288479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff1429fa6e54e65abb5907364fa5f8e', 'request_id': 'req-4495d9e1-f8e6-4970-8aae-e251694a1894'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:15.294691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff1429fa6e54e65abb5907364fa5f8e', 'request_id': 'req-4495d9e1-f8e6-4970-8aae-e251694a1894'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:15.302263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff1429fa6e54e65abb5907364fa5f8e', 'request_id': 'req-4495d9e1-f8e6-4970-8aae-e251694a1894'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:15.313734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] The token cache is being invalidate because project fff1429fa6e54e65abb5907364fa5f8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:15.314030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:15.314684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4495d9e1-f8e6-4970-8aae-e251694a1894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:15.316324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 960/1905] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:15 2022] DELETE /identity/v3/projects/fff1429fa6e54e65abb5907364fa5f8e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:15.322823 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433408c-8e7f-42fe-b74f-0bdce2fbca32 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.323110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433408c-8e7f-42fe-b74f-0bdce2fbca32 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.323388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433408c-8e7f-42fe-b74f-0bdce2fbca32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.324547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:15.342445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d433408c-8e7f-42fe-b74f-0bdce2fbca32 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:15.343648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d433408c-8e7f-42fe-b74f-0bdce2fbca32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:15.349288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cf482-80d9-4233-bb40-ac6b824e5853 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.408130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 961/1906] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:15.416105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee81090-b2e4-440c-b09c-490f96403adc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.416396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee81090-b2e4-440c-b09c-490f96403adc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.416677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee81090-b2e4-440c-b09c-490f96403adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.425003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b1a23-cf2c-4c4a-899e-873d5b7078ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.425993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b1a23-cf2c-4c4a-899e-873d5b7078ae admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.426393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b1a23-cf2c-4c4a-899e-873d5b7078ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.426739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b1a23-cf2c-4c4a-899e-873d5b7078ae admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.427499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b1a23-cf2c-4c4a-899e-873d5b7078ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.430489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b1a23-cf2c-4c4a-899e-873d5b7078ae admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.442529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dee81090-b2e4-440c-b09c-490f96403adc None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:15.444500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dee81090-b2e4-440c-b09c-490f96403adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:15.459826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 946/1907] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:15 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:15.461214 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b1a23-cf2c-4c4a-899e-873d5b7078ae admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.463387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 962/1908] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:15.464707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-946b1a23-cf2c-4c4a-899e-873d5b7078ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:15.468613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee81090-b2e4-440c-b09c-490f96403adc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.480302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee81090-b2e4-440c-b09c-490f96403adc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.483561 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b1a23-cf2c-4c4a-899e-873d5b7078ae admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.490993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77773596-cfba-4d6d-a628-9b423e277498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.491325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76197985-1009-4c46-b801-757d4705261a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.491666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77773596-cfba-4d6d-a628-9b423e277498 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.491948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77773596-cfba-4d6d-a628-9b423e277498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.492052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76197985-1009-4c46-b801-757d4705261a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.492378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77773596-cfba-4d6d-a628-9b423e277498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.492529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76197985-1009-4c46-b801-757d4705261a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.492892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76197985-1009-4c46-b801-757d4705261a admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.493553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77773596-cfba-4d6d-a628-9b423e277498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.493657 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76197985-1009-4c46-b801-757d4705261a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:15.493976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76197985-1009-4c46-b801-757d4705261a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.495177 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77773596-cfba-4d6d-a628-9b423e277498 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.495581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76197985-1009-4c46-b801-757d4705261a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.514950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 947/1909] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:21:15.515120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 963/1910] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:21:15.515120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76197985-1009-4c46-b801-757d4705261a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.515284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77773596-cfba-4d6d-a628-9b423e277498 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.525228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77773596-cfba-4d6d-a628-9b423e277498 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.525623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76197985-1009-4c46-b801-757d4705261a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.529948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a08100-3930-47e0-9a27-0a2cd7e2076a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.530356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305a9a5-c13c-4b4a-9cd5-5d288f9c0f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.530828 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a08100-3930-47e0-9a27-0a2cd7e2076a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.531020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305a9a5-c13c-4b4a-9cd5-5d288f9c0f0f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.531114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a08100-3930-47e0-9a27-0a2cd7e2076a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.531302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305a9a5-c13c-4b4a-9cd5-5d288f9c0f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.531448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a08100-3930-47e0-9a27-0a2cd7e2076a admin admin] PATH_INFO: `/v3/projects/149f4ae356944676aa18733fd30260ba/users/9858b0f9c4c24b99beb8937602914144/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.531575 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305a9a5-c13c-4b4a-9cd5-5d288f9c0f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.532254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305a9a5-c13c-4b4a-9cd5-5d288f9c0f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.534219 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305a9a5-c13c-4b4a-9cd5-5d288f9c0f0f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.548583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a08100-3930-47e0-9a27-0a2cd7e2076a admin admin] RBAC: Authorizing `identity:create_grant(project_id=149f4ae356944676aa18733fd30260ba, user_id=9858b0f9c4c24b99beb8937602914144, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.551373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a08100-3930-47e0-9a27-0a2cd7e2076a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.562883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 964/1911] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:15 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:15.565163 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 948/1912] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:15 2022] PUT /identity/v3/projects/149f4ae356944676aa18733fd30260ba/users/9858b0f9c4c24b99beb8937602914144/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:15.567131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305a9a5-c13c-4b4a-9cd5-5d288f9c0f0f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.569330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a08100-3930-47e0-9a27-0a2cd7e2076a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.638347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a08100-3930-47e0-9a27-0a2cd7e2076a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.639947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:15.643170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4f9c-8ab2-4b61-8168-d5d901ca107d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.643835 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4f9c-8ab2-4b61-8168-d5d901ca107d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.644121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4f9c-8ab2-4b61-8168-d5d901ca107d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.644362 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305a9a5-c13c-4b4a-9cd5-5d288f9c0f0f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.644471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4f9c-8ab2-4b61-8168-d5d901ca107d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.645092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc4f9c-8ab2-4b61-8168-d5d901ca107d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.646702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc4f9c-8ab2-4b61-8168-d5d901ca107d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.650753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e732b-8c37-42d1-8ab0-0b98fd6f2439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.651791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e732b-8c37-42d1-8ab0-0b98fd6f2439 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.652461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e732b-8c37-42d1-8ab0-0b98fd6f2439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.652743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e732b-8c37-42d1-8ab0-0b98fd6f2439 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.653434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e732b-8c37-42d1-8ab0-0b98fd6f2439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.655413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e732b-8c37-42d1-8ab0-0b98fd6f2439 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.657146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 949/1913] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:15.661368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4f9c-8ab2-4b61-8168-d5d901ca107d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.672176 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4f9c-8ab2-4b61-8168-d5d901ca107d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.676758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4112027-bd7b-4b09-9183-9d0be62c8c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.676963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 965/1914] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:15 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:15.677675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4112027-bd7b-4b09-9183-9d0be62c8c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.677990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4112027-bd7b-4b09-9183-9d0be62c8c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.678263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4112027-bd7b-4b09-9183-9d0be62c8c26 admin admin] PATH_INFO: `/v3/projects/149f4ae356944676aa18733fd30260ba/users/9858b0f9c4c24b99beb8937602914144/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.681254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4112027-bd7b-4b09-9183-9d0be62c8c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=149f4ae356944676aa18733fd30260ba, user_id=9858b0f9c4c24b99beb8937602914144, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.681367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e732b-8c37-42d1-8ab0-0b98fd6f2439 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.683920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4112027-bd7b-4b09-9183-9d0be62c8c26 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.691598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e732b-8c37-42d1-8ab0-0b98fd6f2439 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.694832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 950/1915] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:15 2022] PUT /identity/v3/projects/149f4ae356944676aa18733fd30260ba/users/9858b0f9c4c24b99beb8937602914144/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:15.706102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776b1f9-7a36-4687-b17e-3e0d547e5561 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.706399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776b1f9-7a36-4687-b17e-3e0d547e5561 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.706676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776b1f9-7a36-4687-b17e-3e0d547e5561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.734725 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c776b1f9-7a36-4687-b17e-3e0d547e5561 None None] MFA Rules not processed for user `9858b0f9c4c24b99beb8937602914144`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:15.736186 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c776b1f9-7a36-4687-b17e-3e0d547e5561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:15.749957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:15.751677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20b684-ad21-455a-9a72-825f3cb20072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.752562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20b684-ad21-455a-9a72-825f3cb20072 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.753013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20b684-ad21-455a-9a72-825f3cb20072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.753582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20b684-ad21-455a-9a72-825f3cb20072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.754387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20b684-ad21-455a-9a72-825f3cb20072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.756015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20b684-ad21-455a-9a72-825f3cb20072 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.767663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 966/1916] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:21:15.771624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20b684-ad21-455a-9a72-825f3cb20072 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.782039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20b684-ad21-455a-9a72-825f3cb20072 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.787161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c355c44-a1db-4ae6-b4d5-85c3589eecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.788072 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c355c44-a1db-4ae6-b4d5-85c3589eecce admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.788363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c355c44-a1db-4ae6-b4d5-85c3589eecce admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.788647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c355c44-a1db-4ae6-b4d5-85c3589eecce admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.789572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c355c44-a1db-4ae6-b4d5-85c3589eecce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.791220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c355c44-a1db-4ae6-b4d5-85c3589eecce admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.803543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 967/1917] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:21:15.807817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c355c44-a1db-4ae6-b4d5-85c3589eecce admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.820113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c355c44-a1db-4ae6-b4d5-85c3589eecce admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.825850 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 951/1918] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:15.826806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605e30d-93c5-4d9d-bd69-c0a552d66a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.828169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605e30d-93c5-4d9d-bd69-c0a552d66a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.828460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605e30d-93c5-4d9d-bd69-c0a552d66a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.828744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605e30d-93c5-4d9d-bd69-c0a552d66a1e admin admin] PATH_INFO: `/v3/projects/5324e830a70a4dc09baf65e21a6a53bf/users/11cdc3bd092a4bc8930c1d6f3f993624/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.838025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776b1f9-7a36-4687-b17e-3e0d547e5561 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.839564 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c776b1f9-7a36-4687-b17e-3e0d547e5561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:15.846223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605e30d-93c5-4d9d-bd69-c0a552d66a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5324e830a70a4dc09baf65e21a6a53bf, user_id=11cdc3bd092a4bc8930c1d6f3f993624, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:15.848999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605e30d-93c5-4d9d-bd69-c0a552d66a1e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:15.863785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 968/1919] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:15 2022] PUT /identity/v3/projects/5324e830a70a4dc09baf65e21a6a53bf/users/11cdc3bd092a4bc8930c1d6f3f993624/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:15.869198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be70a15-efbd-47df-bf4f-93c8e410af7f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.869498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be70a15-efbd-47df-bf4f-93c8e410af7f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.869816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be70a15-efbd-47df-bf4f-93c8e410af7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.890827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5be70a15-efbd-47df-bf4f-93c8e410af7f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:15.892501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5be70a15-efbd-47df-bf4f-93c8e410af7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:15.904078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776b1f9-7a36-4687-b17e-3e0d547e5561 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:15.961867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9958b-58c2-4910-930c-f1c4692f4f51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:15.962687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9958b-58c2-4910-930c-f1c4692f4f51 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:15.963102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9958b-58c2-4910-930c-f1c4692f4f51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:15.963385 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9958b-58c2-4910-930c-f1c4692f4f51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:15.965421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-88e9958b-58c2-4910-930c-f1c4692f4f51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:15.970355 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 969/1920] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:15.986803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be70a15-efbd-47df-bf4f-93c8e410af7f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:15.988782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5be70a15-efbd-47df-bf4f-93c8e410af7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:16.057486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9958b-58c2-4910-930c-f1c4692f4f51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:16.059407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9958b-58c2-4910-930c-f1c4692f4f51 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:16.065651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:16.069752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be70a15-efbd-47df-bf4f-93c8e410af7f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:16.073865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 952/1921] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:16.085689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5dcfd-a88e-4b64-8259-903d7b258bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:16.087366 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5dcfd-a88e-4b64-8259-903d7b258bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:16.087655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5dcfd-a88e-4b64-8259-903d7b258bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:16.087935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5dcfd-a88e-4b64-8259-903d7b258bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:16.104850 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5dcfd-a88e-4b64-8259-903d7b258bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:16.106346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5dcfd-a88e-4b64-8259-903d7b258bd2 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:16.116720 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 970/1922] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:17.060197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bba4e-9b1d-484c-b532-ee189c8413a2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:17.060689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bba4e-9b1d-484c-b532-ee189c8413a2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:17.060779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bba4e-9b1d-484c-b532-ee189c8413a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:17.089640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-210bba4e-9b1d-484c-b532-ee189c8413a2 None None] MFA Rules not processed for user `11cdc3bd092a4bc8930c1d6f3f993624`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:17.091445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-210bba4e-9b1d-484c-b532-ee189c8413a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:17.099635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:17.165474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 953/1923] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:17.166694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5dcfd-a88e-4b64-8259-903d7b258bd2 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:17.177248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5dcfd-a88e-4b64-8259-903d7b258bd2 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:17.182981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8b95f-5745-4eda-bba4-7d0bce5dd423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:17.183749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8b95f-5745-4eda-bba4-7d0bce5dd423 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:17.184107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8b95f-5745-4eda-bba4-7d0bce5dd423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:17.184455 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8b95f-5745-4eda-bba4-7d0bce5dd423 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:17.185518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8b95f-5745-4eda-bba4-7d0bce5dd423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:17.187988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8b95f-5745-4eda-bba4-7d0bce5dd423 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:17.222281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 971/1924] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:17 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:17.226622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bba4e-9b1d-484c-b532-ee189c8413a2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:17.241355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bba4e-9b1d-484c-b532-ee189c8413a2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:17.331991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed23f88-d497-4eb3-b7f8-58dc0f0f1ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:17.332805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed23f88-d497-4eb3-b7f8-58dc0f0f1ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:17.333222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed23f88-d497-4eb3-b7f8-58dc0f0f1ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:17.333590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed23f88-d497-4eb3-b7f8-58dc0f0f1ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:17.334400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed23f88-d497-4eb3-b7f8-58dc0f0f1ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:17.337478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed23f88-d497-4eb3-b7f8-58dc0f0f1ae0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:17.363193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 954/1925] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:17 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:17.364572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8b95f-5745-4eda-bba4-7d0bce5dd423 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:17.375464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8b95f-5745-4eda-bba4-7d0bce5dd423 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:17.380161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1fa47-6026-4a6f-a3f9-dd1c50e57f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:17.381052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1fa47-6026-4a6f-a3f9-dd1c50e57f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:17.381432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1fa47-6026-4a6f-a3f9-dd1c50e57f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:17.381918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1fa47-6026-4a6f-a3f9-dd1c50e57f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:17.383256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1fa47-6026-4a6f-a3f9-dd1c50e57f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:17.385444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1fa47-6026-4a6f-a3f9-dd1c50e57f2b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:17.396432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 972/1926] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:21:17.405477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed23f88-d497-4eb3-b7f8-58dc0f0f1ae0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:17.417949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed23f88-d497-4eb3-b7f8-58dc0f0f1ae0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:17.421889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fe128-e63d-4a5f-9c72-fc293f3516ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:17.422892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fe128-e63d-4a5f-9c72-fc293f3516ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:17.423276 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fe128-e63d-4a5f-9c72-fc293f3516ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:17.423666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fe128-e63d-4a5f-9c72-fc293f3516ee admin admin] PATH_INFO: `/v3/projects/b42485b433f04674b52150c53078937e/users/62194fe86eaf41e3bf37401172a2e59d/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:17.444694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fe128-e63d-4a5f-9c72-fc293f3516ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42485b433f04674b52150c53078937e, user_id=62194fe86eaf41e3bf37401172a2e59d, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:17.448496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fe128-e63d-4a5f-9c72-fc293f3516ee admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:17.475125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 955/1927] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:17 2022] PUT /identity/v3/projects/b42485b433f04674b52150c53078937e/users/62194fe86eaf41e3bf37401172a2e59d/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:17.479377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1fa47-6026-4a6f-a3f9-dd1c50e57f2b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:17.489413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1fa47-6026-4a6f-a3f9-dd1c50e57f2b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:17.553872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b49e7-0aa9-4ffd-90f3-e0ea0ccba093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:17.555200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b49e7-0aa9-4ffd-90f3-e0ea0ccba093 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:17.555486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b49e7-0aa9-4ffd-90f3-e0ea0ccba093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:17.555762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b49e7-0aa9-4ffd-90f3-e0ea0ccba093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:17.556498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b49e7-0aa9-4ffd-90f3-e0ea0ccba093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:17.558144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b49e7-0aa9-4ffd-90f3-e0ea0ccba093 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:17.568192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 973/1928] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:21:17.572378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fe128-e63d-4a5f-9c72-fc293f3516ee admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:17.582653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fe128-e63d-4a5f-9c72-fc293f3516ee admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:17.587854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735f335-f484-4374-818a-5b24d93120d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:17.588963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735f335-f484-4374-818a-5b24d93120d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:17.589257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735f335-f484-4374-818a-5b24d93120d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:17.589536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735f335-f484-4374-818a-5b24d93120d2 admin admin] PATH_INFO: `/v3/projects/b42485b433f04674b52150c53078937e/users/62194fe86eaf41e3bf37401172a2e59d/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:17.591548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735f335-f484-4374-818a-5b24d93120d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42485b433f04674b52150c53078937e, user_id=62194fe86eaf41e3bf37401172a2e59d, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:17.594235 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735f335-f484-4374-818a-5b24d93120d2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:17.605956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 956/1929] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:17 2022] PUT /identity/v3/projects/b42485b433f04674b52150c53078937e/users/62194fe86eaf41e3bf37401172a2e59d/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:17.958302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b49e7-0aa9-4ffd-90f3-e0ea0ccba093 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:18.062030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b49e7-0aa9-4ffd-90f3-e0ea0ccba093 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:18.164240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0a479-d227-4b2e-a4d2-38325020bb21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:18.164989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0a479-d227-4b2e-a4d2-38325020bb21 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:18.165322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0a479-d227-4b2e-a4d2-38325020bb21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:18.165862 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0a479-d227-4b2e-a4d2-38325020bb21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:18.168065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ceb0a479-d227-4b2e-a4d2-38325020bb21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:18.216642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:18.264000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0a479-d227-4b2e-a4d2-38325020bb21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:18.265748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0a479-d227-4b2e-a4d2-38325020bb21 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:18.281755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 974/1930] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:19.248501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d14c96-9ff2-4406-a420-7e74fd364446 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:19.249131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d14c96-9ff2-4406-a420-7e74fd364446 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:19.249131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d14c96-9ff2-4406-a420-7e74fd364446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:19.287438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e8d14c96-9ff2-4406-a420-7e74fd364446 None None] MFA Rules not processed for user `62194fe86eaf41e3bf37401172a2e59d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:19.290229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e8d14c96-9ff2-4406-a420-7e74fd364446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:19.355897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:19.380004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 957/1931] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:19 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:19.392224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0a479-d227-4b2e-a4d2-38325020bb21 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:19.394369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ceb0a479-d227-4b2e-a4d2-38325020bb21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:19.405343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0a479-d227-4b2e-a4d2-38325020bb21 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:19.409190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95d896-ca6e-4d53-b3ec-a247bfc3776b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:19.409992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95d896-ca6e-4d53-b3ec-a247bfc3776b service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:19.410384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95d896-ca6e-4d53-b3ec-a247bfc3776b service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:19.410814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95d896-ca6e-4d53-b3ec-a247bfc3776b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:19.480827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95d896-ca6e-4d53-b3ec-a247bfc3776b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:19.482476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95d896-ca6e-4d53-b3ec-a247bfc3776b service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:19.491763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 975/1932] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:20.718182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d14c96-9ff2-4406-a420-7e74fd364446 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:20.729225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d14c96-9ff2-4406-a420-7e74fd364446 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:20.732925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1828913-7f31-4a68-b85b-aab1fb1e6b9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:20.733639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1828913-7f31-4a68-b85b-aab1fb1e6b9d service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:20.734014 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1828913-7f31-4a68-b85b-aab1fb1e6b9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:20.734357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1828913-7f31-4a68-b85b-aab1fb1e6b9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:20.806096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1828913-7f31-4a68-b85b-aab1fb1e6b9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:20.807542 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1828913-7f31-4a68-b85b-aab1fb1e6b9d service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:20.817321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 958/1933] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:21.937181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95d896-ca6e-4d53-b3ec-a247bfc3776b service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:21.947844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95d896-ca6e-4d53-b3ec-a247bfc3776b service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:21.955882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d9b83-0cf6-412f-805d-9573d0d7bb0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:21.956858 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d9b83-0cf6-412f-805d-9573d0d7bb0b service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:21.957454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d9b83-0cf6-412f-805d-9573d0d7bb0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:21.957782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d9b83-0cf6-412f-805d-9573d0d7bb0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:21.970116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d9b83-0cf6-412f-805d-9573d0d7bb0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:21.971705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d9b83-0cf6-412f-805d-9573d0d7bb0b service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:21.982766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 976/1934] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:21 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:25.022497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1828913-7f31-4a68-b85b-aab1fb1e6b9d service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:25.083795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1828913-7f31-4a68-b85b-aab1fb1e6b9d service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:25.088997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:25.089930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:25.090213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:25.090487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] PATH_INFO: `/v3/users/c1c0ea9425954868a239b259b11890de` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:25.091661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c0ea9425954868a239b259b11890de)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:25.093650 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:25.141898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c0ea9425954868a239b259b11890de'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:25.149682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c0ea9425954868a239b259b11890de'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:25.150353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c0ea9425954868a239b259b11890de'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:25.165582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c0ea9425954868a239b259b11890de'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:25.174936 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c0ea9425954868a239b259b11890de'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:25.176898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:25.183984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 959/1935] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:25 2022] DELETE /identity/v3/users/c1c0ea9425954868a239b259b11890de => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:25.189022 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d3f08-9202-425d-9c39-b3386e178b81 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:25.189639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d3f08-9202-425d-9c39-b3386e178b81 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:25.189639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d3f08-9202-425d-9c39-b3386e178b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:25.208517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-801d3f08-9202-425d-9c39-b3386e178b81 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:25.209754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-801d3f08-9202-425d-9c39-b3386e178b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:25.280829 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 977/1936] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:25.307759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:25.309305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:25.369680 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44772397-d96b-4a95-ada0-3196f46d2874 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:25.373351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4be08d-ccc5-4134-a097-970b4f3eec46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:25.374042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4be08d-ccc5-4134-a097-970b4f3eec46 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:25.374326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4be08d-ccc5-4134-a097-970b4f3eec46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:25.374606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4be08d-ccc5-4134-a097-970b4f3eec46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:25.384515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4be08d-ccc5-4134-a097-970b4f3eec46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:25.385990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4be08d-ccc5-4134-a097-970b4f3eec46 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:25.402080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 960/1937] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:25.822143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d3f08-9202-425d-9c39-b3386e178b81 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:25.823684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-801d3f08-9202-425d-9c39-b3386e178b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:25.848680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d3f08-9202-425d-9c39-b3386e178b81 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:25.859919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:25.861319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:25.862678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:25.863277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] PATH_INFO: `/v3/projects/60e88317d3164c3b9406608ca9414ce4` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:25.865052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60e88317d3164c3b9406608ca9414ce4)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:25.868076 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:25.993501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:26.002245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60e88317d3164c3b9406608ca9414ce4', 'request_id': 'req-3d81d119-ef63-469f-b59c-de33e27a62d1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.016834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60e88317d3164c3b9406608ca9414ce4', 'request_id': 'req-3d81d119-ef63-469f-b59c-de33e27a62d1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.028098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60e88317d3164c3b9406608ca9414ce4', 'request_id': 'req-3d81d119-ef63-469f-b59c-de33e27a62d1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.033784 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4be08d-ccc5-4134-a097-970b4f3eec46 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:26.035205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ff4be08d-ccc5-4134-a097-970b4f3eec46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:26.045919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] The token cache is being invalidate because project 60e88317d3164c3b9406608ca9414ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:26.046368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.047425 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.049337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 978/1938] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:25 2022] DELETE /identity/v3/projects/60e88317d3164c3b9406608ca9414ce4 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:26.055269 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:26.058928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4be08d-ccc5-4134-a097-970b4f3eec46 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:26.059883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:26.161348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c03fc-64fe-45f1-8123-37615f259d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:26.162386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c03fc-64fe-45f1-8123-37615f259d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:26.163397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c03fc-64fe-45f1-8123-37615f259d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:26.164439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c03fc-64fe-45f1-8123-37615f259d90 admin admin] PATH_INFO: `/v3/users/3c6b2e0f707c46b8af53804616534042` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:26.165923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c03fc-64fe-45f1-8123-37615f259d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c6b2e0f707c46b8af53804616534042)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:26.168529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c03fc-64fe-45f1-8123-37615f259d90 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:26.188646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81d119-ef63-469f-b59c-de33e27a62d1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:26.200272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:26.201018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:26.202088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:26.202495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] PATH_INFO: `/v3/users/8267155806c840ea8590fc00e37cf4f4` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:26.208949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8267155806c840ea8590fc00e37cf4f4)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:26.211654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:26.234605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c4c03fc-64fe-45f1-8123-37615f259d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c6b2e0f707c46b8af53804616534042'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.247957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c4c03fc-64fe-45f1-8123-37615f259d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c6b2e0f707c46b8af53804616534042'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.248621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c4c03fc-64fe-45f1-8123-37615f259d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c6b2e0f707c46b8af53804616534042'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.264100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c4c03fc-64fe-45f1-8123-37615f259d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c6b2e0f707c46b8af53804616534042'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.272503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8267155806c840ea8590fc00e37cf4f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.273553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8267155806c840ea8590fc00e37cf4f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.274401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8267155806c840ea8590fc00e37cf4f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.291410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c4c03fc-64fe-45f1-8123-37615f259d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c6b2e0f707c46b8af53804616534042'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.298966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 961/1939] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:26 2022] DELETE /identity/v3/users/3c6b2e0f707c46b8af53804616534042 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:26.303765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2602b-b980-4f8a-a8ae-066f05c5433b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:26.304089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2602b-b980-4f8a-a8ae-066f05c5433b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:26.304895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2602b-b980-4f8a-a8ae-066f05c5433b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:26.307311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8267155806c840ea8590fc00e37cf4f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.323017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8267155806c840ea8590fc00e37cf4f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.331269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bea2602b-b980-4f8a-a8ae-066f05c5433b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:26.332482 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bea2602b-b980-4f8a-a8ae-066f05c5433b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:26.354403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:26.356996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 979/1940] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:26 2022] DELETE /identity/v3/users/8267155806c840ea8590fc00e37cf4f4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:26.422545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 962/1941] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:26.432979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:26.434889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:26.521236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfac1a9-1c3c-4ad2-ad00-eebec059ce60 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:26.526218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7a076-d5e6-4d9b-bdff-7f3ef5ec8028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:26.527283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7a076-d5e6-4d9b-bdff-7f3ef5ec8028 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:26.528445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7a076-d5e6-4d9b-bdff-7f3ef5ec8028 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:26.528774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7a076-d5e6-4d9b-bdff-7f3ef5ec8028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:26.536779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:26.624760 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7a076-d5e6-4d9b-bdff-7f3ef5ec8028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:26.626668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7a076-d5e6-4d9b-bdff-7f3ef5ec8028 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:26.638477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 980/1942] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:26.808927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2602b-b980-4f8a-a8ae-066f05c5433b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:26.810788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bea2602b-b980-4f8a-a8ae-066f05c5433b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:26.838291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2602b-b980-4f8a-a8ae-066f05c5433b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:26.852390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:26.853491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:26.853811 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:26.854256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] PATH_INFO: `/v3/projects/8de4f875595a4f7185770ed29dc00e00` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:26.856267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8de4f875595a4f7185770ed29dc00e00)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:26.859576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:26.927874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8de4f875595a4f7185770ed29dc00e00', 'request_id': 'req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.938181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8de4f875595a4f7185770ed29dc00e00', 'request_id': 'req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.948398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8de4f875595a4f7185770ed29dc00e00', 'request_id': 'req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.994317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] The token cache is being invalidate because project 8de4f875595a4f7185770ed29dc00e00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:26.994706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.995446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:26.998375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 963/1943] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:26 2022] DELETE /identity/v3/projects/8de4f875595a4f7185770ed29dc00e00 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:27.002635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7a076-d5e6-4d9b-bdff-7f3ef5ec8028 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:27.016474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7a076-d5e6-4d9b-bdff-7f3ef5ec8028 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:27.164298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:27.165498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:27.165821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:27.166114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] PATH_INFO: `/v3/users/9858b0f9c4c24b99beb8937602914144` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:27.167546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9858b0f9c4c24b99beb8937602914144)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:27.171065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:27.228071 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9858b0f9c4c24b99beb8937602914144'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:27.228824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9858b0f9c4c24b99beb8937602914144'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:27.229602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9858b0f9c4c24b99beb8937602914144'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:27.250968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9858b0f9c4c24b99beb8937602914144'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:27.262258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9858b0f9c4c24b99beb8937602914144'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:27.279727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 981/1944] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:27 2022] DELETE /identity/v3/users/9858b0f9c4c24b99beb8937602914144 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:27.468992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:27.470619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:27.490410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c4ff0-0d48-4e1b-a49c-3c788b2de367 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:27.606628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:27.608960 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:27.610195 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:27.610634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] PATH_INFO: `/v3/projects/25e44cd8cfef454eb7e29d2f7daaa917` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:27.613848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e44cd8cfef454eb7e29d2f7daaa917)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:27.617582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:27.697701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:27.699082 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e44cd8cfef454eb7e29d2f7daaa917', 'request_id': 'req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:27.708330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e44cd8cfef454eb7e29d2f7daaa917', 'request_id': 'req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:27.756346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e44cd8cfef454eb7e29d2f7daaa917', 'request_id': 'req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:27.785125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] The token cache is being invalidate because project 25e44cd8cfef454eb7e29d2f7daaa917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:27.785515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:27.786385 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:27.789586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 964/1945] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:27 2022] DELETE /identity/v3/projects/25e44cd8cfef454eb7e29d2f7daaa917 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:27.793433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:27.794950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:27.816372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d43d8-5a3f-41f4-9cb8-d5056e9b613a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:27.863280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:27.916526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741df8d1-b032-4df1-b78d-1fa759050398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:27.917588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741df8d1-b032-4df1-b78d-1fa759050398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:27.918175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741df8d1-b032-4df1-b78d-1fa759050398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:27.918175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741df8d1-b032-4df1-b78d-1fa759050398 admin admin] PATH_INFO: `/v3/users/4c6a54c70a1a489c8baf861059744830` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:27.919419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741df8d1-b032-4df1-b78d-1fa759050398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c6a54c70a1a489c8baf861059744830)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:27.922202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741df8d1-b032-4df1-b78d-1fa759050398 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.001907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-741df8d1-b032-4df1-b78d-1fa759050398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c6a54c70a1a489c8baf861059744830'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.002814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-741df8d1-b032-4df1-b78d-1fa759050398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c6a54c70a1a489c8baf861059744830'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.004047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-741df8d1-b032-4df1-b78d-1fa759050398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c6a54c70a1a489c8baf861059744830'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.039861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-741df8d1-b032-4df1-b78d-1fa759050398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c6a54c70a1a489c8baf861059744830'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.054895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:28.058599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:28.066418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-741df8d1-b032-4df1-b78d-1fa759050398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c6a54c70a1a489c8baf861059744830'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.090443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 982/1946] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:27 2022] DELETE /identity/v3/users/4c6a54c70a1a489c8baf861059744830 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:28.091616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dee7de-20f5-45cb-8352-5bee7b8c2fb2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:28.161768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:28.162990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.163350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.163734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] PATH_INFO: `/v3/projects/149f4ae356944676aa18733fd30260ba` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.165561 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=149f4ae356944676aa18733fd30260ba)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:28.167568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.230515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149f4ae356944676aa18733fd30260ba', 'request_id': 'req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.237456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149f4ae356944676aa18733fd30260ba', 'request_id': 'req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.244747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149f4ae356944676aa18733fd30260ba', 'request_id': 'req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.265055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] The token cache is being invalidate because project 149f4ae356944676aa18733fd30260ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:28.265388 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.270392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-79f66a1d-1c80-4315-a0bc-759d1c37d6d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.272037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 965/1947] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:28 2022] DELETE /identity/v3/projects/149f4ae356944676aa18733fd30260ba => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:28.281527 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df88e6-6543-4ac1-9b72-61c1f6bb85bc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.282029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df88e6-6543-4ac1-9b72-61c1f6bb85bc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.282225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df88e6-6543-4ac1-9b72-61c1f6bb85bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.301844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a2df88e6-6543-4ac1-9b72-61c1f6bb85bc None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:28.303117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a2df88e6-6543-4ac1-9b72-61c1f6bb85bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:28.368665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 983/1948] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:28.376648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b54c6-a743-488f-86bd-85a040b5fb6f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.376939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b54c6-a743-488f-86bd-85a040b5fb6f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.377225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b54c6-a743-488f-86bd-85a040b5fb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.396330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9b6b54c6-a743-488f-86bd-85a040b5fb6f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:28.397537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9b6b54c6-a743-488f-86bd-85a040b5fb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:28.405968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 966/1949] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:28.410237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df88e6-6543-4ac1-9b72-61c1f6bb85bc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:28.421224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df88e6-6543-4ac1-9b72-61c1f6bb85bc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:28.428861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc6ee4-ce7b-4d08-9560-1ae7c426596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:28.430165 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc6ee4-ce7b-4d08-9560-1ae7c426596e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.430507 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc6ee4-ce7b-4d08-9560-1ae7c426596e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.430851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc6ee4-ce7b-4d08-9560-1ae7c426596e admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.431827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc6ee4-ce7b-4d08-9560-1ae7c426596e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:28.432146 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc6ee4-ce7b-4d08-9560-1ae7c426596e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:28.445382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc6ee4-ce7b-4d08-9560-1ae7c426596e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.472828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 984/1950] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:21:28.474104 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b54c6-a743-488f-86bd-85a040b5fb6f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:28.476952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9b6b54c6-a743-488f-86bd-85a040b5fb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:28.477961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc6ee4-ce7b-4d08-9560-1ae7c426596e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:28.488916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc6ee4-ce7b-4d08-9560-1ae7c426596e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:28.495406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48efc8-0824-4445-8a0f-8833a1121968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:28.495642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b54c6-a743-488f-86bd-85a040b5fb6f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:28.496287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48efc8-0824-4445-8a0f-8833a1121968 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.496590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48efc8-0824-4445-8a0f-8833a1121968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.496946 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48efc8-0824-4445-8a0f-8833a1121968 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.497743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48efc8-0824-4445-8a0f-8833a1121968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:28.499805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48efc8-0824-4445-8a0f-8833a1121968 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.500930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:28.501879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.502108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.502381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] PATH_INFO: `/v3/projects/d374f54812904e219fdf7be7a33bfe71` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.503574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d374f54812904e219fdf7be7a33bfe71)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:28.505541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.530280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 985/1951] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:28 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:28.534190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48efc8-0824-4445-8a0f-8833a1121968 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:28.544481 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:28.556680 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd374f54812904e219fdf7be7a33bfe71', 'request_id': 'req-32eea9ae-2856-49d8-9932-637992da468b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.569153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:28.572001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd374f54812904e219fdf7be7a33bfe71', 'request_id': 'req-32eea9ae-2856-49d8-9932-637992da468b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.580925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd374f54812904e219fdf7be7a33bfe71', 'request_id': 'req-32eea9ae-2856-49d8-9932-637992da468b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.597134 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] The token cache is being invalidate because project d374f54812904e219fdf7be7a33bfe71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:28.597630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.598559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32eea9ae-2856-49d8-9932-637992da468b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:28.600450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 967/1952] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:28 2022] DELETE /identity/v3/projects/d374f54812904e219fdf7be7a33bfe71 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:28.606443 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc211e65-215a-44ca-b141-b2634e625f3d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.606831 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc211e65-215a-44ca-b141-b2634e625f3d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.607248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc211e65-215a-44ca-b141-b2634e625f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.609323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48efc8-0824-4445-8a0f-8833a1121968 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:28.627585 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cc211e65-215a-44ca-b141-b2634e625f3d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:28.628800 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cc211e65-215a-44ca-b141-b2634e625f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:28.685824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78b3af-86be-4251-800c-84878c3f5776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:28.686534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78b3af-86be-4251-800c-84878c3f5776 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.686821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78b3af-86be-4251-800c-84878c3f5776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.687101 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78b3af-86be-4251-800c-84878c3f5776 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.687810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78b3af-86be-4251-800c-84878c3f5776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:28.690063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78b3af-86be-4251-800c-84878c3f5776 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.715944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 968/1953] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:28.721285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 986/1954] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:28 2022] POST /identity/v3/users => generated 295 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:28.725469 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc211e65-215a-44ca-b141-b2634e625f3d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:28.726829 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493272b6-998e-4fad-b159-39e731f14b7e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.727191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493272b6-998e-4fad-b159-39e731f14b7e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.727535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493272b6-998e-4fad-b159-39e731f14b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.728220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cc211e65-215a-44ca-b141-b2634e625f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:28.742594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc211e65-215a-44ca-b141-b2634e625f3d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:28.748516 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad80f45-82f3-466d-9326-c8680b46d51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:28.749235 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad80f45-82f3-466d-9326-c8680b46d51d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.749351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-493272b6-998e-4fad-b159-39e731f14b7e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:28.749518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad80f45-82f3-466d-9326-c8680b46d51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.749822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad80f45-82f3-466d-9326-c8680b46d51d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.750544 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-493272b6-998e-4fad-b159-39e731f14b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:28.750884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad80f45-82f3-466d-9326-c8680b46d51d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:28.752519 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad80f45-82f3-466d-9326-c8680b46d51d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.759613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 987/1955] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:28.763756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493272b6-998e-4fad-b159-39e731f14b7e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:28.765429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 969/1956] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:28.770217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad80f45-82f3-466d-9326-c8680b46d51d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:28.774959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493272b6-998e-4fad-b159-39e731f14b7e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:28.779949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4b0c6-cd91-4cb7-adf8-82f730735a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:28.780747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4b0c6-cd91-4cb7-adf8-82f730735a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.781123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4b0c6-cd91-4cb7-adf8-82f730735a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.781479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4b0c6-cd91-4cb7-adf8-82f730735a75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.782266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad80f45-82f3-466d-9326-c8680b46d51d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:28.782404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4b0c6-cd91-4cb7-adf8-82f730735a75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:28.782757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4b0c6-cd91-4cb7-adf8-82f730735a75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:28.784479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4b0c6-cd91-4cb7-adf8-82f730735a75 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.789145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ddfaf-fc33-4c0a-a788-4d715700efe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:28.790081 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ddfaf-fc33-4c0a-a788-4d715700efe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.790439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ddfaf-fc33-4c0a-a788-4d715700efe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.790787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ddfaf-fc33-4c0a-a788-4d715700efe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.791596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ddfaf-fc33-4c0a-a788-4d715700efe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:28.793574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ddfaf-fc33-4c0a-a788-4d715700efe3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.803088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 988/1957] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:21:28.804827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4b0c6-cd91-4cb7-adf8-82f730735a75 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:28.808942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 970/1958] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:28.809968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ddfaf-fc33-4c0a-a788-4d715700efe3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:28.819135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4b0c6-cd91-4cb7-adf8-82f730735a75 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:28.819866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ddfaf-fc33-4c0a-a788-4d715700efe3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:28.825429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115f9cf-992f-4ae9-8e3d-1c8369647ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:28.826369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e95f5-078f-4767-880a-efc55d8c6679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:28.826525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115f9cf-992f-4ae9-8e3d-1c8369647ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.826855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115f9cf-992f-4ae9-8e3d-1c8369647ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.827195 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115f9cf-992f-4ae9-8e3d-1c8369647ce2 admin admin] PATH_INFO: `/v3/projects/079d5c88293d48e682e922fd6b2faade/users/a7b165eab1ed4da2ad3b7d31459ce40c/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.827308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e95f5-078f-4767-880a-efc55d8c6679 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.827595 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e95f5-078f-4767-880a-efc55d8c6679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.827957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e95f5-078f-4767-880a-efc55d8c6679 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.828717 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e95f5-078f-4767-880a-efc55d8c6679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:28.831430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e95f5-078f-4767-880a-efc55d8c6679 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.844211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115f9cf-992f-4ae9-8e3d-1c8369647ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079d5c88293d48e682e922fd6b2faade, user_id=a7b165eab1ed4da2ad3b7d31459ce40c, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:28.846925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115f9cf-992f-4ae9-8e3d-1c8369647ce2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.860212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 989/1959] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:28 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:28.861354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 971/1960] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:28 2022] PUT /identity/v3/projects/079d5c88293d48e682e922fd6b2faade/users/a7b165eab1ed4da2ad3b7d31459ce40c/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:28.864456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e95f5-078f-4767-880a-efc55d8c6679 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:28.874011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81767b66-a3d5-49c2-b301-5d8df25bd3ee None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.874217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81767b66-a3d5-49c2-b301-5d8df25bd3ee None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.875391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81767b66-a3d5-49c2-b301-5d8df25bd3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.919887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-81767b66-a3d5-49c2-b301-5d8df25bd3ee None None] MFA Rules not processed for user `a7b165eab1ed4da2ad3b7d31459ce40c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:28.921186 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-81767b66-a3d5-49c2-b301-5d8df25bd3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:28.957915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e95f5-078f-4767-880a-efc55d8c6679 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:28.964451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc15af6-d9a0-430f-ba1a-2ddc0ebe290b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:28.965358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15af6-d9a0-430f-ba1a-2ddc0ebe290b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:28.965978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15af6-d9a0-430f-ba1a-2ddc0ebe290b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:28.966323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15af6-d9a0-430f-ba1a-2ddc0ebe290b admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:28.967869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc15af6-d9a0-430f-ba1a-2ddc0ebe290b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:28.970369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc15af6-d9a0-430f-ba1a-2ddc0ebe290b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:28.980479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:29.001819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 990/1961] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:28 2022] POST /identity/v3/users => generated 284 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:29.007099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc15af6-d9a0-430f-ba1a-2ddc0ebe290b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:29.012113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 972/1962] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:29.023210 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc15af6-d9a0-430f-ba1a-2ddc0ebe290b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:29.025211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81767b66-a3d5-49c2-b301-5d8df25bd3ee None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:29.027603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-81767b66-a3d5-49c2-b301-5d8df25bd3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:29.029203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620ff26-69d5-4c02-a691-6d2d55373142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:29.030161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620ff26-69d5-4c02-a691-6d2d55373142 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.030522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620ff26-69d5-4c02-a691-6d2d55373142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.030865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620ff26-69d5-4c02-a691-6d2d55373142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:29.031680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620ff26-69d5-4c02-a691-6d2d55373142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:29.033558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620ff26-69d5-4c02-a691-6d2d55373142 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:29.049506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 991/1963] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:29.055052 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620ff26-69d5-4c02-a691-6d2d55373142 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:29.067598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620ff26-69d5-4c02-a691-6d2d55373142 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:29.074205 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274e7d9-026c-41df-b33c-5912270f276d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:29.074887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274e7d9-026c-41df-b33c-5912270f276d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.075174 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274e7d9-026c-41df-b33c-5912270f276d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.075456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274e7d9-026c-41df-b33c-5912270f276d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:29.076195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274e7d9-026c-41df-b33c-5912270f276d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:29.077839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274e7d9-026c-41df-b33c-5912270f276d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:29.094900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 992/1964] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:29.096241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274e7d9-026c-41df-b33c-5912270f276d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:29.110399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274e7d9-026c-41df-b33c-5912270f276d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:29.115563 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:29.115864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712aa4a-eeb3-42e9-ad78-b17f028253e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:29.116880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712aa4a-eeb3-42e9-ad78-b17f028253e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.117165 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712aa4a-eeb3-42e9-ad78-b17f028253e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.117442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712aa4a-eeb3-42e9-ad78-b17f028253e3 admin admin] PATH_INFO: `/v3/projects/2b6d1e6b5fc4418a9cf5034a25b9ac44/users/5288201a8e404659ab5fd18e4c18a1d3/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:29.141381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712aa4a-eeb3-42e9-ad78-b17f028253e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b6d1e6b5fc4418a9cf5034a25b9ac44, user_id=5288201a8e404659ab5fd18e4c18a1d3, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:29.146246 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712aa4a-eeb3-42e9-ad78-b17f028253e3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:29.159199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81767b66-a3d5-49c2-b301-5d8df25bd3ee None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:29.166620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 993/1965] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:29 2022] PUT /identity/v3/projects/2b6d1e6b5fc4418a9cf5034a25b9ac44/users/5288201a8e404659ab5fd18e4c18a1d3/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:29.176627 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fab5c-c69c-423f-9b39-7ba1a64443d9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.176960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fab5c-c69c-423f-9b39-7ba1a64443d9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.177433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fab5c-c69c-423f-9b39-7ba1a64443d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:29.251751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-970fab5c-c69c-423f-9b39-7ba1a64443d9 None None] MFA Rules not processed for user `5288201a8e404659ab5fd18e4c18a1d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:29.253698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-970fab5c-c69c-423f-9b39-7ba1a64443d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:29.255356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e286c82-0297-4760-b795-aaccfcead36e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:29.256140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e286c82-0297-4760-b795-aaccfcead36e service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.256431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e286c82-0297-4760-b795-aaccfcead36e service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.256715 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e286c82-0297-4760-b795-aaccfcead36e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:29.340925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 994/1966] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:29.345409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fab5c-c69c-423f-9b39-7ba1a64443d9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:29.348942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e286c82-0297-4760-b795-aaccfcead36e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:29.350764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e286c82-0297-4760-b795-aaccfcead36e service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:29.365652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 973/1967] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:29.442195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fab5c-c69c-423f-9b39-7ba1a64443d9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:29.449261 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93daff-d44a-4daa-aa7d-18053d8cc721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:29.449993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93daff-d44a-4daa-aa7d-18053d8cc721 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.450284 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93daff-d44a-4daa-aa7d-18053d8cc721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.450560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93daff-d44a-4daa-aa7d-18053d8cc721 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:29.451325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93daff-d44a-4daa-aa7d-18053d8cc721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:29.453259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93daff-d44a-4daa-aa7d-18053d8cc721 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:29.485225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 995/1968] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:29 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:29.489132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e286c82-0297-4760-b795-aaccfcead36e service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:29.502214 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e286c82-0297-4760-b795-aaccfcead36e service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:29.591456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41214de6-b5e1-4027-b7c6-b03ee51afb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:29.592829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41214de6-b5e1-4027-b7c6-b03ee51afb3e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.593287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41214de6-b5e1-4027-b7c6-b03ee51afb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.593641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41214de6-b5e1-4027-b7c6-b03ee51afb3e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:29.595961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41214de6-b5e1-4027-b7c6-b03ee51afb3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:29.596814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41214de6-b5e1-4027-b7c6-b03ee51afb3e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:29.628646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 974/1969] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:29 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:29.637811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93daff-d44a-4daa-aa7d-18053d8cc721 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:29.658235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93daff-d44a-4daa-aa7d-18053d8cc721 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:29.666078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f319c7-3eee-4729-9024-7585867d1ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:29.666836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f319c7-3eee-4729-9024-7585867d1ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.667123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f319c7-3eee-4729-9024-7585867d1ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.667401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f319c7-3eee-4729-9024-7585867d1ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:29.668133 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f319c7-3eee-4729-9024-7585867d1ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:29.669758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f319c7-3eee-4729-9024-7585867d1ae0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:29.683051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 996/1970] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:29.687396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41214de6-b5e1-4027-b7c6-b03ee51afb3e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:29.718785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41214de6-b5e1-4027-b7c6-b03ee51afb3e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:29.728494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bce354-e0e3-44e0-981d-76ecdf0b3b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:29.728494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce354-e0e3-44e0-981d-76ecdf0b3b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.728494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce354-e0e3-44e0-981d-76ecdf0b3b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.728494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce354-e0e3-44e0-981d-76ecdf0b3b93 admin admin] PATH_INFO: `/v3/projects/6bc56edc6a6c4ac28d71ac13c5261361/users/73ae55ac8e35443a87c5fcf312912721/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:29.762101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bce354-e0e3-44e0-981d-76ecdf0b3b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bc56edc6a6c4ac28d71ac13c5261361, user_id=73ae55ac8e35443a87c5fcf312912721, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:29.767767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bce354-e0e3-44e0-981d-76ecdf0b3b93 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:29.797336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 975/1971] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:29 2022] PUT /identity/v3/projects/6bc56edc6a6c4ac28d71ac13c5261361/users/73ae55ac8e35443a87c5fcf312912721/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:29.801016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f319c7-3eee-4729-9024-7585867d1ae0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:29.816120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f319c7-3eee-4729-9024-7585867d1ae0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:29.827938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:29.900864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ef479-01f5-4f80-a5fa-6601cfbdbea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:29.901707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ef479-01f5-4f80-a5fa-6601cfbdbea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.902029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ef479-01f5-4f80-a5fa-6601cfbdbea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.902889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ef479-01f5-4f80-a5fa-6601cfbdbea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:29.903872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ef479-01f5-4f80-a5fa-6601cfbdbea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:29.905378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ef479-01f5-4f80-a5fa-6601cfbdbea5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:29.922879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 997/1972] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:29.924117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bce354-e0e3-44e0-981d-76ecdf0b3b93 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:29.931366 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:29.940112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bce354-e0e3-44e0-981d-76ecdf0b3b93 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:29.951077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c7376-e9fe-490b-bb1b-5fd911dd9d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:29.952307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c7376-e9fe-490b-bb1b-5fd911dd9d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.952666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c7376-e9fe-490b-bb1b-5fd911dd9d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.953020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c7376-e9fe-490b-bb1b-5fd911dd9d19 admin admin] PATH_INFO: `/v3/projects/6bc56edc6a6c4ac28d71ac13c5261361/users/73ae55ac8e35443a87c5fcf312912721/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:29.955367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c7376-e9fe-490b-bb1b-5fd911dd9d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bc56edc6a6c4ac28d71ac13c5261361, user_id=73ae55ac8e35443a87c5fcf312912721, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:29.958336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c7376-e9fe-490b-bb1b-5fd911dd9d19 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:29.971416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 976/1973] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:29 2022] PUT /identity/v3/projects/6bc56edc6a6c4ac28d71ac13c5261361/users/73ae55ac8e35443a87c5fcf312912721/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:29.982682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d90f-9c5b-432a-96e0-fc747a2a625b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:29.983019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d90f-9c5b-432a-96e0-fc747a2a625b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:29.983312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d90f-9c5b-432a-96e0-fc747a2a625b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:30.012948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e609d90f-9c5b-432a-96e0-fc747a2a625b None None] MFA Rules not processed for user `73ae55ac8e35443a87c5fcf312912721`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:30.014173 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e609d90f-9c5b-432a-96e0-fc747a2a625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:30.117214 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 998/1974] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:29 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:30.129236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c7376-e9fe-490b-bb1b-5fd911dd9d19 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:30.131019 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1b6c7376-e9fe-490b-bb1b-5fd911dd9d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:30.194466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c7376-e9fe-490b-bb1b-5fd911dd9d19 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:30.198853 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0051d7a-62e1-40f2-8afe-0c1b9b3e871d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:30.199530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0051d7a-62e1-40f2-8afe-0c1b9b3e871d service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:30.199813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0051d7a-62e1-40f2-8afe-0c1b9b3e871d service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:30.200090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0051d7a-62e1-40f2-8afe-0c1b9b3e871d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:30.210635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0051d7a-62e1-40f2-8afe-0c1b9b3e871d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:30.212628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0051d7a-62e1-40f2-8afe-0c1b9b3e871d service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:30.223353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 977/1975] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:30 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:30.272187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609d90f-9c5b-432a-96e0-fc747a2a625b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:30.282648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609d90f-9c5b-432a-96e0-fc747a2a625b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:30.286326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3382be9-9657-4ba2-adb9-0242d3f5c9d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:30.286977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3382be9-9657-4ba2-adb9-0242d3f5c9d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:30.287382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3382be9-9657-4ba2-adb9-0242d3f5c9d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:30.287952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3382be9-9657-4ba2-adb9-0242d3f5c9d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:30.347248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3382be9-9657-4ba2-adb9-0242d3f5c9d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:30.348721 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3382be9-9657-4ba2-adb9-0242d3f5c9d7 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:30.359274 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 999/1976] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:33.823835 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0051d7a-62e1-40f2-8afe-0c1b9b3e871d service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:33.889522 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0051d7a-62e1-40f2-8afe-0c1b9b3e871d service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:33.894204 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:33.895099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:33.895447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:33.895736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] PATH_INFO: `/v3/users/5288201a8e404659ab5fd18e4c18a1d3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:33.897551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5288201a8e404659ab5fd18e4c18a1d3)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:33.901668 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:33.957619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5288201a8e404659ab5fd18e4c18a1d3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:33.966659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5288201a8e404659ab5fd18e4c18a1d3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:33.967375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5288201a8e404659ab5fd18e4c18a1d3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:33.982504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5288201a8e404659ab5fd18e4c18a1d3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:33.989483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5288201a8e404659ab5fd18e4c18a1d3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:33.995699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 978/1977] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:33 2022] DELETE /identity/v3/users/5288201a8e404659ab5fd18e4c18a1d3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:34.000484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd9141-6271-4790-b31c-33e12da9a3ca None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:34.001002 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd9141-6271-4790-b31c-33e12da9a3ca None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:34.001110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd9141-6271-4790-b31c-33e12da9a3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:34.018650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cecd9141-6271-4790-b31c-33e12da9a3ca None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:34.019868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cecd9141-6271-4790-b31c-33e12da9a3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:34.047266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:34.089122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1000/1978] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:34.095775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:34.097236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:34.120063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:34.173235 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378f69a-50c1-4029-a668-c17609a0a491 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:34.177008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95f4d7-cd03-4c3a-a6c4-f14fcbca8104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:34.177662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95f4d7-cd03-4c3a-a6c4-f14fcbca8104 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:34.178109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95f4d7-cd03-4c3a-a6c4-f14fcbca8104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:34.178394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95f4d7-cd03-4c3a-a6c4-f14fcbca8104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:34.188571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95f4d7-cd03-4c3a-a6c4-f14fcbca8104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:34.190119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95f4d7-cd03-4c3a-a6c4-f14fcbca8104 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:34.207182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 979/1979] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:34.503118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd9141-6271-4790-b31c-33e12da9a3ca None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:34.504541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cecd9141-6271-4790-b31c-33e12da9a3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:34.516795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd9141-6271-4790-b31c-33e12da9a3ca None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:34.521572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:34.522571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:34.522896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:34.523291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] PATH_INFO: `/v3/projects/2b6d1e6b5fc4418a9cf5034a25b9ac44` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:34.524549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b6d1e6b5fc4418a9cf5034a25b9ac44)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:34.526676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:34.575249 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b6d1e6b5fc4418a9cf5034a25b9ac44', 'request_id': 'req-ce060b04-5efd-4f49-99c8-b32c7d39e850'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:34.581642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b6d1e6b5fc4418a9cf5034a25b9ac44', 'request_id': 'req-ce060b04-5efd-4f49-99c8-b32c7d39e850'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:34.589669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b6d1e6b5fc4418a9cf5034a25b9ac44', 'request_id': 'req-ce060b04-5efd-4f49-99c8-b32c7d39e850'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:34.600230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] The token cache is being invalidate because project 2b6d1e6b5fc4418a9cf5034a25b9ac44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:34.600548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:34.601324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:34.602809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1001/1980] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:34 2022] DELETE /identity/v3/projects/2b6d1e6b5fc4418a9cf5034a25b9ac44 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:34.606740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95f4d7-cd03-4c3a-a6c4-f14fcbca8104 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:34.616132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95f4d7-cd03-4c3a-a6c4-f14fcbca8104 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:34.672589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a7ab5-1228-46d2-8e8b-2928b1207551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:34.673520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a7ab5-1228-46d2-8e8b-2928b1207551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:34.673782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a7ab5-1228-46d2-8e8b-2928b1207551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:34.674065 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a7ab5-1228-46d2-8e8b-2928b1207551 admin admin] PATH_INFO: `/v3/users/73ae55ac8e35443a87c5fcf312912721` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:34.675287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a7ab5-1228-46d2-8e8b-2928b1207551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ae55ac8e35443a87c5fcf312912721)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:34.677311 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a7ab5-1228-46d2-8e8b-2928b1207551 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:34.728908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f21a7ab5-1228-46d2-8e8b-2928b1207551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ae55ac8e35443a87c5fcf312912721'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:34.737109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f21a7ab5-1228-46d2-8e8b-2928b1207551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ae55ac8e35443a87c5fcf312912721'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:34.738016 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f21a7ab5-1228-46d2-8e8b-2928b1207551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ae55ac8e35443a87c5fcf312912721'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:34.750942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f21a7ab5-1228-46d2-8e8b-2928b1207551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ae55ac8e35443a87c5fcf312912721'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:34.758201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f21a7ab5-1228-46d2-8e8b-2928b1207551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ae55ac8e35443a87c5fcf312912721'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:34.764222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 980/1981] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:34 2022] DELETE /identity/v3/users/73ae55ac8e35443a87c5fcf312912721 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:35.171537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:35.173045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:35.187652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce060b04-5efd-4f49-99c8-b32c7d39e850 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:35.247783 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:35.248651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:35.248930 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:35.249203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] PATH_INFO: `/v3/projects/6bc56edc6a6c4ac28d71ac13c5261361` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:35.250421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bc56edc6a6c4ac28d71ac13c5261361)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:35.252451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:35.318250 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bc56edc6a6c4ac28d71ac13c5261361', 'request_id': 'req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:35.326503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bc56edc6a6c4ac28d71ac13c5261361', 'request_id': 'req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:35.330014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:35.339383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bc56edc6a6c4ac28d71ac13c5261361', 'request_id': 'req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:35.355225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] The token cache is being invalidate because project 6bc56edc6a6c4ac28d71ac13c5261361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:35.355585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:35.356291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03f99bda-d83a-4aaf-8f0a-cacc9a4959e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:35.357947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1002/1982] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:35 2022] DELETE /identity/v3/projects/6bc56edc6a6c4ac28d71ac13c5261361 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:35.363142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a959c77-3309-4c4c-8a75-f4e07bbc43af None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:35.363692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a959c77-3309-4c4c-8a75-f4e07bbc43af None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:35.363780 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a959c77-3309-4c4c-8a75-f4e07bbc43af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:35.389940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2a959c77-3309-4c4c-8a75-f4e07bbc43af None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:35.391217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2a959c77-3309-4c4c-8a75-f4e07bbc43af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:35.429481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:35.474303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 981/1983] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:35.482867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b678e3-3d92-415e-8d0d-337843f2e547 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:35.483164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b678e3-3d92-415e-8d0d-337843f2e547 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:35.483462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b678e3-3d92-415e-8d0d-337843f2e547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:35.504553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-06b678e3-3d92-415e-8d0d-337843f2e547 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:35.506320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-06b678e3-3d92-415e-8d0d-337843f2e547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:35.516533 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1003/1984] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:35.521555 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a959c77-3309-4c4c-8a75-f4e07bbc43af None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:35.533902 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a959c77-3309-4c4c-8a75-f4e07bbc43af None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:35.541140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e6189-2743-4d29-88cc-cc38d96cece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:35.541859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e6189-2743-4d29-88cc-cc38d96cece5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:35.542150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e6189-2743-4d29-88cc-cc38d96cece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:35.542413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e6189-2743-4d29-88cc-cc38d96cece5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:35.543164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e6189-2743-4d29-88cc-cc38d96cece5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:35.543497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e6189-2743-4d29-88cc-cc38d96cece5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:35.545147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e6189-2743-4d29-88cc-cc38d96cece5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:35.558868 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 982/1985] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:21:35.563196 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b678e3-3d92-415e-8d0d-337843f2e547 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:35.577662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b678e3-3d92-415e-8d0d-337843f2e547 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:35.582568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc15592-24d3-4d18-8c14-54947d0ac4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:35.583310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc15592-24d3-4d18-8c14-54947d0ac4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:35.583564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc15592-24d3-4d18-8c14-54947d0ac4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:35.583843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc15592-24d3-4d18-8c14-54947d0ac4c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:35.584534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc15592-24d3-4d18-8c14-54947d0ac4c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:35.587429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc15592-24d3-4d18-8c14-54947d0ac4c5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:35.616063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1004/1986] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:35 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:35.620327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e6189-2743-4d29-88cc-cc38d96cece5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:35.631953 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e6189-2743-4d29-88cc-cc38d96cece5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:35.693684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adda0cf-10cf-460b-a375-95e36815526b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:35.694563 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adda0cf-10cf-460b-a375-95e36815526b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:35.695110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adda0cf-10cf-460b-a375-95e36815526b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:35.695522 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adda0cf-10cf-460b-a375-95e36815526b admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:35.696922 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adda0cf-10cf-460b-a375-95e36815526b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:35.700364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adda0cf-10cf-460b-a375-95e36815526b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:35.724300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 983/1987] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:35 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:35.729962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc15592-24d3-4d18-8c14-54947d0ac4c5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:35.740918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc15592-24d3-4d18-8c14-54947d0ac4c5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:35.746698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501e9be-c849-4b50-a87e-7a4f18945c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:35.747473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e9be-c849-4b50-a87e-7a4f18945c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:35.747825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e9be-c849-4b50-a87e-7a4f18945c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:35.748164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e9be-c849-4b50-a87e-7a4f18945c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:35.748965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501e9be-c849-4b50-a87e-7a4f18945c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:35.750725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501e9be-c849-4b50-a87e-7a4f18945c03 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:35.766213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1005/1988] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:35.769812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adda0cf-10cf-460b-a375-95e36815526b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:35.780966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adda0cf-10cf-460b-a375-95e36815526b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:35.788581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86842218-8d5b-415c-bf1e-d666d427e006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:35.788581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86842218-8d5b-415c-bf1e-d666d427e006 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:35.788581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86842218-8d5b-415c-bf1e-d666d427e006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:35.788581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86842218-8d5b-415c-bf1e-d666d427e006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:35.789264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86842218-8d5b-415c-bf1e-d666d427e006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:35.790942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86842218-8d5b-415c-bf1e-d666d427e006 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:35.803858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 984/1989] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:35.807980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501e9be-c849-4b50-a87e-7a4f18945c03 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:35.818968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501e9be-c849-4b50-a87e-7a4f18945c03 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:35.824382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99fe28-cd10-43a2-a608-37a853ea23d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:35.825517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99fe28-cd10-43a2-a608-37a853ea23d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:35.825930 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99fe28-cd10-43a2-a608-37a853ea23d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:35.826172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99fe28-cd10-43a2-a608-37a853ea23d1 admin admin] PATH_INFO: `/v3/projects/6104a1a3feed4f109b45489280782944/users/da1660c6d2544b798e7e688d874baf2a/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:35.843030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99fe28-cd10-43a2-a608-37a853ea23d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6104a1a3feed4f109b45489280782944, user_id=da1660c6d2544b798e7e688d874baf2a, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:35.845493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99fe28-cd10-43a2-a608-37a853ea23d1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:35.860815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1006/1990] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:35 2022] PUT /identity/v3/projects/6104a1a3feed4f109b45489280782944/users/da1660c6d2544b798e7e688d874baf2a/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:35.872370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3411f-904d-4fa7-9d80-db52f3d8d5df None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:35.872664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3411f-904d-4fa7-9d80-db52f3d8d5df None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:35.872943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3411f-904d-4fa7-9d80-db52f3d8d5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:35.902207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-60e3411f-904d-4fa7-9d80-db52f3d8d5df None None] MFA Rules not processed for user `da1660c6d2544b798e7e688d874baf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:35.903441 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-60e3411f-904d-4fa7-9d80-db52f3d8d5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:35.969565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 985/1991] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:35.974133 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99fe28-cd10-43a2-a608-37a853ea23d1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:36.053286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99fe28-cd10-43a2-a608-37a853ea23d1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:36.059928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0854689-1bf5-4a74-a1fc-314ef6f6c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:36.062486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0854689-1bf5-4a74-a1fc-314ef6f6c23a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:36.063053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0854689-1bf5-4a74-a1fc-314ef6f6c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:36.063431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0854689-1bf5-4a74-a1fc-314ef6f6c23a admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:36.064618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0854689-1bf5-4a74-a1fc-314ef6f6c23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:36.067843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0854689-1bf5-4a74-a1fc-314ef6f6c23a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:36.096647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1007/1992] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:35 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:36.101844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3411f-904d-4fa7-9d80-db52f3d8d5df None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:36.118319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3411f-904d-4fa7-9d80-db52f3d8d5df None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:36.153492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:36.192667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29679db0-4b74-4702-b4c3-a53fae45ca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:36.193366 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679db0-4b74-4702-b4c3-a53fae45ca27 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:36.194096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679db0-4b74-4702-b4c3-a53fae45ca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:36.194096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679db0-4b74-4702-b4c3-a53fae45ca27 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:36.194866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29679db0-4b74-4702-b4c3-a53fae45ca27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:36.197050 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29679db0-4b74-4702-b4c3-a53fae45ca27 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:36.227136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 986/1993] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:36 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:36.227258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0854689-1bf5-4a74-a1fc-314ef6f6c23a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:36.237312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0854689-1bf5-4a74-a1fc-314ef6f6c23a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:36.246280 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b69dc-9866-40cd-a3ec-0e4a5f1b2e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:36.246952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b69dc-9866-40cd-a3ec-0e4a5f1b2e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:36.247231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b69dc-9866-40cd-a3ec-0e4a5f1b2e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:36.247507 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b69dc-9866-40cd-a3ec-0e4a5f1b2e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:36.248237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b69dc-9866-40cd-a3ec-0e4a5f1b2e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:36.249890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b69dc-9866-40cd-a3ec-0e4a5f1b2e47 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:36.262136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1008/1994] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:21:36.266036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29679db0-4b74-4702-b4c3-a53fae45ca27 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:36.277335 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29679db0-4b74-4702-b4c3-a53fae45ca27 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:36.282268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08c26f-28c9-4d64-a1aa-d6adbffad06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:36.283171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08c26f-28c9-4d64-a1aa-d6adbffad06e admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:36.283442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08c26f-28c9-4d64-a1aa-d6adbffad06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:36.283713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08c26f-28c9-4d64-a1aa-d6adbffad06e admin admin] PATH_INFO: `/v3/projects/b4e24e49a8b74947be8bb801279c2cf3/users/54a25c655a654048980dae8d116cd580/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:36.306205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08c26f-28c9-4d64-a1aa-d6adbffad06e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e24e49a8b74947be8bb801279c2cf3, user_id=54a25c655a654048980dae8d116cd580, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:36.309278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08c26f-28c9-4d64-a1aa-d6adbffad06e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:36.327349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 987/1995] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:36 2022] PUT /identity/v3/projects/b4e24e49a8b74947be8bb801279c2cf3/users/54a25c655a654048980dae8d116cd580/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:36.331545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b69dc-9866-40cd-a3ec-0e4a5f1b2e47 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:36.343467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b69dc-9866-40cd-a3ec-0e4a5f1b2e47 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:36.406116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:36.421479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37e784-e1b3-4341-9acc-ad81b2ed1342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:36.422209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37e784-e1b3-4341-9acc-ad81b2ed1342 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:36.422555 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37e784-e1b3-4341-9acc-ad81b2ed1342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:36.423048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37e784-e1b3-4341-9acc-ad81b2ed1342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:36.424169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37e784-e1b3-4341-9acc-ad81b2ed1342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:36.426068 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37e784-e1b3-4341-9acc-ad81b2ed1342 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:36.440002 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1009/1996] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:36.446975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08c26f-28c9-4d64-a1aa-d6adbffad06e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:36.461273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08c26f-28c9-4d64-a1aa-d6adbffad06e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:36.466545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24e934-29dd-40f1-9362-71bb1b7e7705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:36.467451 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e934-29dd-40f1-9362-71bb1b7e7705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:36.467762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e934-29dd-40f1-9362-71bb1b7e7705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:36.468013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e934-29dd-40f1-9362-71bb1b7e7705 admin admin] PATH_INFO: `/v3/projects/b4e24e49a8b74947be8bb801279c2cf3/users/54a25c655a654048980dae8d116cd580/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:36.470221 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24e934-29dd-40f1-9362-71bb1b7e7705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e24e49a8b74947be8bb801279c2cf3, user_id=54a25c655a654048980dae8d116cd580, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:36.473004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24e934-29dd-40f1-9362-71bb1b7e7705 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:36.486519 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 988/1997] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:36 2022] PUT /identity/v3/projects/b4e24e49a8b74947be8bb801279c2cf3/users/54a25c655a654048980dae8d116cd580/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:36.496629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16085b-e592-4a2c-9d61-ae3c096e0e11 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:36.496924 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16085b-e592-4a2c-9d61-ae3c096e0e11 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:36.497208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16085b-e592-4a2c-9d61-ae3c096e0e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:36.527537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0b16085b-e592-4a2c-9d61-ae3c096e0e11 None None] MFA Rules not processed for user `54a25c655a654048980dae8d116cd580`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:36.528873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0b16085b-e592-4a2c-9d61-ae3c096e0e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:36.610130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1010/1998] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:36 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:36.622112 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24e934-29dd-40f1-9362-71bb1b7e7705 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:36.623819 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ff24e934-29dd-40f1-9362-71bb1b7e7705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:36.682341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24e934-29dd-40f1-9362-71bb1b7e7705 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:36.686038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f3c7e-a671-4109-bd44-7313746fc132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:36.686684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f3c7e-a671-4109-bd44-7313746fc132 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:36.686962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f3c7e-a671-4109-bd44-7313746fc132 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:36.687258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f3c7e-a671-4109-bd44-7313746fc132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:36.697601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f3c7e-a671-4109-bd44-7313746fc132 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:36.699079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f3c7e-a671-4109-bd44-7313746fc132 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:36.715211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 989/1999] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:36 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:36.799318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16085b-e592-4a2c-9d61-ae3c096e0e11 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:36.809923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16085b-e592-4a2c-9d61-ae3c096e0e11 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:36.812673 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9611f-4e2d-42a2-a919-af9352842380 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:36.812971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9611f-4e2d-42a2-a919-af9352842380 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:36.813250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9611f-4e2d-42a2-a919-af9352842380 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:36.814678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 990/2000] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:21:36.817595 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5f023-65e0-44a4-8a2c-eb4858a069fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:36.818279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5f023-65e0-44a4-8a2c-eb4858a069fa service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:36.818562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5f023-65e0-44a4-8a2c-eb4858a069fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:36.818842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5f023-65e0-44a4-8a2c-eb4858a069fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:36.832688 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e257af7-de0a-4dc3-a1bf-9937ea3c43d5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:36.833145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e257af7-de0a-4dc3-a1bf-9937ea3c43d5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:36.833430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e257af7-de0a-4dc3-a1bf-9937ea3c43d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:36.856897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5e257af7-de0a-4dc3-a1bf-9937ea3c43d5 None None] MFA Rules not processed for user `6719421144734ef49b3809df30a45961`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:36.858319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5e257af7-de0a-4dc3-a1bf-9937ea3c43d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:36.892632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5f023-65e0-44a4-8a2c-eb4858a069fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:36.894375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5f023-65e0-44a4-8a2c-eb4858a069fa service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:36.909188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1011/2001] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:36.915836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5f023-65e0-44a4-8a2c-eb4858a069fa service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:36.917480 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2bb5f023-65e0-44a4-8a2c-eb4858a069fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:36.963181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 991/2002] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:21:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:36.979686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e257af7-de0a-4dc3-a1bf-9937ea3c43d5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:36.980793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5e257af7-de0a-4dc3-a1bf-9937ea3c43d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:36.997949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e257af7-de0a-4dc3-a1bf-9937ea3c43d5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:37.003252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5f023-65e0-44a4-8a2c-eb4858a069fa service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:37.004775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f445fa-79c9-429e-9a82-638cabaa287e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:37.005815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f445fa-79c9-429e-9a82-638cabaa287e service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.006101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f445fa-79c9-429e-9a82-638cabaa287e service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.006387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f445fa-79c9-429e-9a82-638cabaa287e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.009729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef1aea-ccc4-4c7d-b50a-d8757bc64dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:37.012283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1aea-ccc4-4c7d-b50a-d8757bc64dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.012683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1aea-ccc4-4c7d-b50a-d8757bc64dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.012846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1aea-ccc4-4c7d-b50a-d8757bc64dfd admin admin] PATH_INFO: `/v3/users/18cd966ad655466e85930f7c505fb360` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.014120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef1aea-ccc4-4c7d-b50a-d8757bc64dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=18cd966ad655466e85930f7c505fb360)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:37.016179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f445fa-79c9-429e-9a82-638cabaa287e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:37.018042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef1aea-ccc4-4c7d-b50a-d8757bc64dfd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:37.018216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f445fa-79c9-429e-9a82-638cabaa287e service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:37.031528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 992/2003] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:37.035275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:37.077011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50ef1aea-ccc4-4c7d-b50a-d8757bc64dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18cd966ad655466e85930f7c505fb360'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:37.077707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50ef1aea-ccc4-4c7d-b50a-d8757bc64dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18cd966ad655466e85930f7c505fb360'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:37.078394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50ef1aea-ccc4-4c7d-b50a-d8757bc64dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18cd966ad655466e85930f7c505fb360'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:37.092180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50ef1aea-ccc4-4c7d-b50a-d8757bc64dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18cd966ad655466e85930f7c505fb360'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:37.100441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50ef1aea-ccc4-4c7d-b50a-d8757bc64dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18cd966ad655466e85930f7c505fb360'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:37.115302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1012/2004] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:36 2022] DELETE /identity/v3/users/18cd966ad655466e85930f7c505fb360 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:37.303054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f445fa-79c9-429e-9a82-638cabaa287e service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:37.315871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f445fa-79c9-429e-9a82-638cabaa287e service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:37.321027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:37.376526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:37.377390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.377669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.377970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] PATH_INFO: `/v3/projects/9b5ee0d2e59647dc98721de1b8dfb4db` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.379200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b5ee0d2e59647dc98721de1b8dfb4db)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:37.381130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:37.434738 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b5ee0d2e59647dc98721de1b8dfb4db', 'request_id': 'req-58d44f6e-da16-4887-a17c-ccd764ef6573'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:37.440167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b5ee0d2e59647dc98721de1b8dfb4db', 'request_id': 'req-58d44f6e-da16-4887-a17c-ccd764ef6573'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:37.447261 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b5ee0d2e59647dc98721de1b8dfb4db', 'request_id': 'req-58d44f6e-da16-4887-a17c-ccd764ef6573'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:37.460996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] The token cache is being invalidate because project 9b5ee0d2e59647dc98721de1b8dfb4db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:37.461458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:37.462117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-58d44f6e-da16-4887-a17c-ccd764ef6573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:37.463677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 993/2005] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:37 2022] DELETE /identity/v3/projects/9b5ee0d2e59647dc98721de1b8dfb4db => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:37.468684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f0c4d-345a-4ac3-9806-368aad6dfdb8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.469147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f0c4d-345a-4ac3-9806-368aad6dfdb8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.469256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f0c4d-345a-4ac3-9806-368aad6dfdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.487538 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-955f0c4d-345a-4ac3-9806-368aad6dfdb8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:37.488864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-955f0c4d-345a-4ac3-9806-368aad6dfdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:37.551091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1013/2006] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:37.558867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b010fe3-07a1-421d-b9dc-77d7d1f45365 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.559157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b010fe3-07a1-421d-b9dc-77d7d1f45365 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.559441 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b010fe3-07a1-421d-b9dc-77d7d1f45365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.579378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0b010fe3-07a1-421d-b9dc-77d7d1f45365 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:37.580567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0b010fe3-07a1-421d-b9dc-77d7d1f45365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:37.589580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 994/2007] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:37.593814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f0c4d-345a-4ac3-9806-368aad6dfdb8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:37.603736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f0c4d-345a-4ac3-9806-368aad6dfdb8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:37.608487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90b9c7-b264-4d7e-b340-06dd2de889ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:37.609145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90b9c7-b264-4d7e-b340-06dd2de889ab admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.609426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90b9c7-b264-4d7e-b340-06dd2de889ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.609697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90b9c7-b264-4d7e-b340-06dd2de889ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.610443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90b9c7-b264-4d7e-b340-06dd2de889ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:37.610769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90b9c7-b264-4d7e-b340-06dd2de889ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:37.612352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90b9c7-b264-4d7e-b340-06dd2de889ab admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:37.624790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1014/2008] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:21:37.628992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b010fe3-07a1-421d-b9dc-77d7d1f45365 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:37.638765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b010fe3-07a1-421d-b9dc-77d7d1f45365 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:37.643381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc709e96-5e5b-4fa2-8199-29bed4d6b647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:37.644028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709e96-5e5b-4fa2-8199-29bed4d6b647 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.644312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709e96-5e5b-4fa2-8199-29bed4d6b647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.644590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709e96-5e5b-4fa2-8199-29bed4d6b647 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.645277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc709e96-5e5b-4fa2-8199-29bed4d6b647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:37.647228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc709e96-5e5b-4fa2-8199-29bed4d6b647 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:37.673297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 995/2009] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:37 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:37.677694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90b9c7-b264-4d7e-b340-06dd2de889ab admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:37.688135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90b9c7-b264-4d7e-b340-06dd2de889ab admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:37.743959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcc4bb-acb8-479a-9035-83d68cdbf5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:37.744623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcc4bb-acb8-479a-9035-83d68cdbf5bb admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.744905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcc4bb-acb8-479a-9035-83d68cdbf5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.745180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcc4bb-acb8-479a-9035-83d68cdbf5bb admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.745905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcc4bb-acb8-479a-9035-83d68cdbf5bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:37.747820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcc4bb-acb8-479a-9035-83d68cdbf5bb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:37.768062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1015/2010] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:37 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:37.772127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc709e96-5e5b-4fa2-8199-29bed4d6b647 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:37.782887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc709e96-5e5b-4fa2-8199-29bed4d6b647 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:37.787488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2034be-301b-4a1b-b183-04bee5a84a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:37.788426 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2034be-301b-4a1b-b183-04bee5a84a2f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.788705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2034be-301b-4a1b-b183-04bee5a84a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.788983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2034be-301b-4a1b-b183-04bee5a84a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.790358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2034be-301b-4a1b-b183-04bee5a84a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:37.791962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2034be-301b-4a1b-b183-04bee5a84a2f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:37.802222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 996/2011] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:37.806665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcc4bb-acb8-479a-9035-83d68cdbf5bb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:37.816839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcc4bb-acb8-479a-9035-83d68cdbf5bb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:37.822258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52afe63-8052-4fea-9724-0c95ef4dd601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:37.822923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52afe63-8052-4fea-9724-0c95ef4dd601 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.823230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52afe63-8052-4fea-9724-0c95ef4dd601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.823488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52afe63-8052-4fea-9724-0c95ef4dd601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.824230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52afe63-8052-4fea-9724-0c95ef4dd601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:37.825859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52afe63-8052-4fea-9724-0c95ef4dd601 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:37.827883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:37.840824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1016/2012] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:37.842001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2034be-301b-4a1b-b183-04bee5a84a2f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:37.852362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2034be-301b-4a1b-b183-04bee5a84a2f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:37.857132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af994ebd-d2d5-4522-b17f-741839dfd6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:37.858056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994ebd-d2d5-4522-b17f-741839dfd6bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.858341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994ebd-d2d5-4522-b17f-741839dfd6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.858615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994ebd-d2d5-4522-b17f-741839dfd6bb admin admin] PATH_INFO: `/v3/projects/7a02f436f9a34b29b069ca4e5551f8df/users/569a192bd1084b379aa40ceab7fb1c31/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.873733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af994ebd-d2d5-4522-b17f-741839dfd6bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a02f436f9a34b29b069ca4e5551f8df, user_id=569a192bd1084b379aa40ceab7fb1c31, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:37.876406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af994ebd-d2d5-4522-b17f-741839dfd6bb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:37.889565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 997/2013] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:37 2022] PUT /identity/v3/projects/7a02f436f9a34b29b069ca4e5551f8df/users/569a192bd1084b379aa40ceab7fb1c31/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:37.899742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0d9c4-2102-4d73-9a55-2c5ad7a9975d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:37.900143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0d9c4-2102-4d73-9a55-2c5ad7a9975d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:37.900514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0d9c4-2102-4d73-9a55-2c5ad7a9975d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:37.927025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-74d0d9c4-2102-4d73-9a55-2c5ad7a9975d None None] MFA Rules not processed for user `569a192bd1084b379aa40ceab7fb1c31`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:37.928234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-74d0d9c4-2102-4d73-9a55-2c5ad7a9975d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:37.996594 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1017/2014] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:38.008562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af994ebd-d2d5-4522-b17f-741839dfd6bb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:38.010307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-af994ebd-d2d5-4522-b17f-741839dfd6bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:38.052731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:38.095324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af994ebd-d2d5-4522-b17f-741839dfd6bb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:38.102290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e102d8-ff24-4bcf-984e-d1228ee7d92f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:38.103153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e102d8-ff24-4bcf-984e-d1228ee7d92f service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:38.103462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e102d8-ff24-4bcf-984e-d1228ee7d92f service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:38.103964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e102d8-ff24-4bcf-984e-d1228ee7d92f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:38.113095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e102d8-ff24-4bcf-984e-d1228ee7d92f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:38.114719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e102d8-ff24-4bcf-984e-d1228ee7d92f service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:38.127470 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 998/2015] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:41.320323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5279-0f38-4e45-83f3-4efc9b6833e5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:41.320851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5279-0f38-4e45-83f3-4efc9b6833e5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:41.320975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5279-0f38-4e45-83f3-4efc9b6833e5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:41.322545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1018/2016] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:21:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:21:41.333244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fd2f1-0686-4c74-bf8f-16f02160fbfc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:41.333830 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fd2f1-0686-4c74-bf8f-16f02160fbfc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:41.334164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fd2f1-0686-4c74-bf8f-16f02160fbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:41.355513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-417fd2f1-0686-4c74-bf8f-16f02160fbfc None None] MFA Rules not processed for user `6719421144734ef49b3809df30a45961`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:41.356939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-417fd2f1-0686-4c74-bf8f-16f02160fbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:41.434269 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 999/2017] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:21:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:41.446024 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc5279-0f38-4e45-83f3-4efc9b6833e5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:41.447890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-11bc5279-0f38-4e45-83f3-4efc9b6833e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:41.465067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc5279-0f38-4e45-83f3-4efc9b6833e5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:41.470491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc986c-3a0c-464d-afb1-d1d4486c60c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:41.471211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc986c-3a0c-464d-afb1-d1d4486c60c1 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:41.471566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc986c-3a0c-464d-afb1-d1d4486c60c1 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:41.471910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc986c-3a0c-464d-afb1-d1d4486c60c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:41.483649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc986c-3a0c-464d-afb1-d1d4486c60c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:41.485364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc986c-3a0c-464d-afb1-d1d4486c60c1 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:41.497158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1019/2018] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:41.969507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fd2f1-0686-4c74-bf8f-16f02160fbfc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:41.970973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-417fd2f1-0686-4c74-bf8f-16f02160fbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:42.061478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fd2f1-0686-4c74-bf8f-16f02160fbfc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:42.066787 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:42.067666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:42.067950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:42.068240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] PATH_INFO: `/v3/users/11cdc3bd092a4bc8930c1d6f3f993624` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:42.069425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11cdc3bd092a4bc8930c1d6f3f993624)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:42.071904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:42.122997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11cdc3bd092a4bc8930c1d6f3f993624'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.130513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11cdc3bd092a4bc8930c1d6f3f993624'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.131056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11cdc3bd092a4bc8930c1d6f3f993624'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.133524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:42.148007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11cdc3bd092a4bc8930c1d6f3f993624'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.155455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11cdc3bd092a4bc8930c1d6f3f993624'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.161943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1000/2019] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:41 2022] DELETE /identity/v3/users/11cdc3bd092a4bc8930c1d6f3f993624 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:42.347515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc986c-3a0c-464d-afb1-d1d4486c60c1 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:42.361911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc986c-3a0c-464d-afb1-d1d4486c60c1 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:42.429878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:42.430888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:42.431175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:42.431729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] PATH_INFO: `/v3/projects/5324e830a70a4dc09baf65e21a6a53bf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:42.433090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5324e830a70a4dc09baf65e21a6a53bf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:42.435546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:42.488867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5324e830a70a4dc09baf65e21a6a53bf', 'request_id': 'req-2865e424-56eb-4b90-9459-a9958feed72e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.497000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5324e830a70a4dc09baf65e21a6a53bf', 'request_id': 'req-2865e424-56eb-4b90-9459-a9958feed72e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.505205 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5324e830a70a4dc09baf65e21a6a53bf', 'request_id': 'req-2865e424-56eb-4b90-9459-a9958feed72e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.517117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] The token cache is being invalidate because project 5324e830a70a4dc09baf65e21a6a53bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:42.517411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.518135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.519706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1020/2020] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:42 2022] DELETE /identity/v3/projects/5324e830a70a4dc09baf65e21a6a53bf => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:42.524480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:42.525990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:42.540074 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ce7e1-990b-44fe-9945-7bdd79679316 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:42.602984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6db66f-9571-4229-8074-a4656802d19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:42.604046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6db66f-9571-4229-8074-a4656802d19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:42.604391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6db66f-9571-4229-8074-a4656802d19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:42.604804 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6db66f-9571-4229-8074-a4656802d19c admin admin] PATH_INFO: `/v3/users/62194fe86eaf41e3bf37401172a2e59d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:42.606605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6db66f-9571-4229-8074-a4656802d19c admin admin] RBAC: Authorizing `identity:delete_user(user_id=62194fe86eaf41e3bf37401172a2e59d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:42.608659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6db66f-9571-4229-8074-a4656802d19c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:42.657805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fe6db66f-9571-4229-8074-a4656802d19c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62194fe86eaf41e3bf37401172a2e59d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.666085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fe6db66f-9571-4229-8074-a4656802d19c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62194fe86eaf41e3bf37401172a2e59d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.666926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fe6db66f-9571-4229-8074-a4656802d19c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62194fe86eaf41e3bf37401172a2e59d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.680397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fe6db66f-9571-4229-8074-a4656802d19c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62194fe86eaf41e3bf37401172a2e59d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.688056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fe6db66f-9571-4229-8074-a4656802d19c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62194fe86eaf41e3bf37401172a2e59d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:42.694145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1001/2021] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:42 2022] DELETE /identity/v3/users/62194fe86eaf41e3bf37401172a2e59d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:42.948974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:42.950422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:42.961653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:42.967653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865e424-56eb-4b90-9459-a9958feed72e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:43.028653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:43.029535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.029843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.030167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] PATH_INFO: `/v3/projects/b42485b433f04674b52150c53078937e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.031354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b42485b433f04674b52150c53078937e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:43.033268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:43.083806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b42485b433f04674b52150c53078937e', 'request_id': 'req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:43.090715 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b42485b433f04674b52150c53078937e', 'request_id': 'req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:43.098494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b42485b433f04674b52150c53078937e', 'request_id': 'req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:43.109962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] The token cache is being invalidate because project b42485b433f04674b52150c53078937e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:43.110259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:43.111245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-59e66a01-68b1-4fdf-b9cf-9fc489b1b0af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:43.112780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1021/2022] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:42 2022] DELETE /identity/v3/projects/b42485b433f04674b52150c53078937e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:43.117897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329727f-0441-4498-b5e8-1860d0b8d810 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.118192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329727f-0441-4498-b5e8-1860d0b8d810 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.118473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329727f-0441-4498-b5e8-1860d0b8d810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.135825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b329727f-0441-4498-b5e8-1860d0b8d810 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:43.137027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b329727f-0441-4498-b5e8-1860d0b8d810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:43.186467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:43.202412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1002/2023] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:43.211320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8380ab8-47ac-45ed-90dd-0554601099de None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.211613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8380ab8-47ac-45ed-90dd-0554601099de None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.211895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8380ab8-47ac-45ed-90dd-0554601099de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.230618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f8380ab8-47ac-45ed-90dd-0554601099de None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:43.231882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f8380ab8-47ac-45ed-90dd-0554601099de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:43.240902 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1022/2024] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:43.245790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329727f-0441-4498-b5e8-1860d0b8d810 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:43.255669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329727f-0441-4498-b5e8-1860d0b8d810 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:43.260279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a0fe9-ab15-4ded-9a1d-f90847fa55da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:43.260939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a0fe9-ab15-4ded-9a1d-f90847fa55da admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.261223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a0fe9-ab15-4ded-9a1d-f90847fa55da admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.261501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a0fe9-ab15-4ded-9a1d-f90847fa55da admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.262255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a0fe9-ab15-4ded-9a1d-f90847fa55da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:43.262565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a0fe9-ab15-4ded-9a1d-f90847fa55da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:43.264200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a0fe9-ab15-4ded-9a1d-f90847fa55da admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:43.275634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1003/2025] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:21:43.279964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8380ab8-47ac-45ed-90dd-0554601099de None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:43.290457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8380ab8-47ac-45ed-90dd-0554601099de None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:43.295391 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d575bf-02a6-44b4-b77b-e8a4313f69a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:43.296054 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d575bf-02a6-44b4-b77b-e8a4313f69a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.296358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d575bf-02a6-44b4-b77b-e8a4313f69a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.296657 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d575bf-02a6-44b4-b77b-e8a4313f69a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.297367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d575bf-02a6-44b4-b77b-e8a4313f69a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:43.299372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d575bf-02a6-44b4-b77b-e8a4313f69a2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:43.325612 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1023/2026] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:43 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:43.329832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a0fe9-ab15-4ded-9a1d-f90847fa55da admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:43.339737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a0fe9-ab15-4ded-9a1d-f90847fa55da admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:43.396729 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221b789-c6a3-42dc-92b6-3833d00d5ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:43.397490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221b789-c6a3-42dc-92b6-3833d00d5ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.397813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221b789-c6a3-42dc-92b6-3833d00d5ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.398130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221b789-c6a3-42dc-92b6-3833d00d5ed0 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.398860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221b789-c6a3-42dc-92b6-3833d00d5ed0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:43.401073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221b789-c6a3-42dc-92b6-3833d00d5ed0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:43.424401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1004/2027] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:43 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:43.429247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d575bf-02a6-44b4-b77b-e8a4313f69a2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:43.439751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d575bf-02a6-44b4-b77b-e8a4313f69a2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:43.444655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789d406-920f-4e7a-b894-9e5cd8ba711f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:43.445334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789d406-920f-4e7a-b894-9e5cd8ba711f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.445618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789d406-920f-4e7a-b894-9e5cd8ba711f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.445966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789d406-920f-4e7a-b894-9e5cd8ba711f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.446696 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789d406-920f-4e7a-b894-9e5cd8ba711f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:43.448291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789d406-920f-4e7a-b894-9e5cd8ba711f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:43.459398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1024/2028] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:43.463489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221b789-c6a3-42dc-92b6-3833d00d5ed0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:43.473550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221b789-c6a3-42dc-92b6-3833d00d5ed0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:43.479077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ad7c-2b0e-4bc4-a9ed-76971b2dfbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:43.479872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ad7c-2b0e-4bc4-a9ed-76971b2dfbde admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.480187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ad7c-2b0e-4bc4-a9ed-76971b2dfbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.480540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ad7c-2b0e-4bc4-a9ed-76971b2dfbde admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.481329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88ad7c-2b0e-4bc4-a9ed-76971b2dfbde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:43.482995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88ad7c-2b0e-4bc4-a9ed-76971b2dfbde admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:43.494234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1005/2029] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:43.498753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789d406-920f-4e7a-b894-9e5cd8ba711f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:43.509011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789d406-920f-4e7a-b894-9e5cd8ba711f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:43.513607 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551a752-3733-4dc3-b068-4b8ae7b2ba60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:43.514509 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551a752-3733-4dc3-b068-4b8ae7b2ba60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.514792 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551a752-3733-4dc3-b068-4b8ae7b2ba60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.515063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551a752-3733-4dc3-b068-4b8ae7b2ba60 admin admin] PATH_INFO: `/v3/projects/9cd8ebb622b04615ad8c2661bc83ca28/users/03c9333ca2da43fda0b422641c6226ab/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.529313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7551a752-3733-4dc3-b068-4b8ae7b2ba60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd8ebb622b04615ad8c2661bc83ca28, user_id=03c9333ca2da43fda0b422641c6226ab, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:43.532043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7551a752-3733-4dc3-b068-4b8ae7b2ba60 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:43.546642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1025/2030] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:43 2022] PUT /identity/v3/projects/9cd8ebb622b04615ad8c2661bc83ca28/users/03c9333ca2da43fda0b422641c6226ab/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:43.557629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca742044-d2d7-4010-bc7c-ea2492b171cc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.558066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca742044-d2d7-4010-bc7c-ea2492b171cc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.558229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca742044-d2d7-4010-bc7c-ea2492b171cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.586897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ca742044-d2d7-4010-bc7c-ea2492b171cc None None] MFA Rules not processed for user `03c9333ca2da43fda0b422641c6226ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:43.588121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ca742044-d2d7-4010-bc7c-ea2492b171cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:43.652794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1006/2031] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:43.657259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551a752-3733-4dc3-b068-4b8ae7b2ba60 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:43.692902 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:43.730946 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551a752-3733-4dc3-b068-4b8ae7b2ba60 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:43.738323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4abaa-a503-4e2b-aaa0-3986758bb231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:43.739294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4abaa-a503-4e2b-aaa0-3986758bb231 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.739722 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4abaa-a503-4e2b-aaa0-3986758bb231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.740377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4abaa-a503-4e2b-aaa0-3986758bb231 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.741816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4abaa-a503-4e2b-aaa0-3986758bb231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:43.744246 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4abaa-a503-4e2b-aaa0-3986758bb231 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:43.772799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1026/2032] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:43 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:43.776886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca742044-d2d7-4010-bc7c-ea2492b171cc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:43.788306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca742044-d2d7-4010-bc7c-ea2492b171cc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:43.832411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:43.845396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb7730-ac74-4a84-8352-67883caffcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:43.846104 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb7730-ac74-4a84-8352-67883caffcda admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.846410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb7730-ac74-4a84-8352-67883caffcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.846695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb7730-ac74-4a84-8352-67883caffcda admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.847376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb7730-ac74-4a84-8352-67883caffcda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:43.849298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb7730-ac74-4a84-8352-67883caffcda admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:43.872410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1007/2033] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:43 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:43.873649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4abaa-a503-4e2b-aaa0-3986758bb231 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:43.885112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4abaa-a503-4e2b-aaa0-3986758bb231 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:43.889814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57335ba6-8200-4699-8800-f4768e27d758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:43.890476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57335ba6-8200-4699-8800-f4768e27d758 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.890763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57335ba6-8200-4699-8800-f4768e27d758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.891042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57335ba6-8200-4699-8800-f4768e27d758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.891774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57335ba6-8200-4699-8800-f4768e27d758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:43.893376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57335ba6-8200-4699-8800-f4768e27d758 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:43.905488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1027/2034] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:21:43.909984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb7730-ac74-4a84-8352-67883caffcda admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:43.920527 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb7730-ac74-4a84-8352-67883caffcda admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:43.926207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8536b-e893-476e-8cc5-b18bf493d34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:43.927097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8536b-e893-476e-8cc5-b18bf493d34e admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:43.927499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8536b-e893-476e-8cc5-b18bf493d34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:43.927780 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8536b-e893-476e-8cc5-b18bf493d34e admin admin] PATH_INFO: `/v3/projects/b560904a5aca42ee9bdcf3c7a0cc25ed/users/8991b28d60ec4bb68dfa7035f53037dc/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:43.943548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8536b-e893-476e-8cc5-b18bf493d34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b560904a5aca42ee9bdcf3c7a0cc25ed, user_id=8991b28d60ec4bb68dfa7035f53037dc, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:43.946852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8536b-e893-476e-8cc5-b18bf493d34e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:43.962355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1008/2035] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:43 2022] PUT /identity/v3/projects/b560904a5aca42ee9bdcf3c7a0cc25ed/users/8991b28d60ec4bb68dfa7035f53037dc/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:43.966208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57335ba6-8200-4699-8800-f4768e27d758 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:43.978850 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57335ba6-8200-4699-8800-f4768e27d758 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:44.065283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fe920-8285-42c9-a756-370aa9ffc28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:44.066193 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fe920-8285-42c9-a756-370aa9ffc28e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:44.066484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fe920-8285-42c9-a756-370aa9ffc28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:44.066768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fe920-8285-42c9-a756-370aa9ffc28e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:44.067675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6fe920-8285-42c9-a756-370aa9ffc28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:44.070268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6fe920-8285-42c9-a756-370aa9ffc28e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:44.095954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1028/2036] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:44.100139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8536b-e893-476e-8cc5-b18bf493d34e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:44.112294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8536b-e893-476e-8cc5-b18bf493d34e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:44.118087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdd11c-e5a7-4ac9-8cca-41b2a1d79917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:44.119162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdd11c-e5a7-4ac9-8cca-41b2a1d79917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:44.119347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdd11c-e5a7-4ac9-8cca-41b2a1d79917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:44.119624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdd11c-e5a7-4ac9-8cca-41b2a1d79917 admin admin] PATH_INFO: `/v3/projects/b560904a5aca42ee9bdcf3c7a0cc25ed/users/8991b28d60ec4bb68dfa7035f53037dc/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:44.121707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdd11c-e5a7-4ac9-8cca-41b2a1d79917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b560904a5aca42ee9bdcf3c7a0cc25ed, user_id=8991b28d60ec4bb68dfa7035f53037dc, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:44.124322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdd11c-e5a7-4ac9-8cca-41b2a1d79917 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:44.137749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1009/2037] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:44 2022] PUT /identity/v3/projects/b560904a5aca42ee9bdcf3c7a0cc25ed/users/8991b28d60ec4bb68dfa7035f53037dc/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:44.148354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9bbd1-fbb5-4492-ab5a-136afefe877a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:44.148354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9bbd1-fbb5-4492-ab5a-136afefe877a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:44.148535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9bbd1-fbb5-4492-ab5a-136afefe877a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:44.176091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6ce9bbd1-fbb5-4492-ab5a-136afefe877a None None] MFA Rules not processed for user `8991b28d60ec4bb68dfa7035f53037dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:44.177389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6ce9bbd1-fbb5-4492-ab5a-136afefe877a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:44.256508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1029/2038] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:44.264286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdd11c-e5a7-4ac9-8cca-41b2a1d79917 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:44.266360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-30fdd11c-e5a7-4ac9-8cca-41b2a1d79917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:44.326710 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdd11c-e5a7-4ac9-8cca-41b2a1d79917 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:44.330392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bdc88-e9d6-4e30-ac58-e736eaa7579f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:44.331078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bdc88-e9d6-4e30-ac58-e736eaa7579f service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:44.331387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bdc88-e9d6-4e30-ac58-e736eaa7579f service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:44.331665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bdc88-e9d6-4e30-ac58-e736eaa7579f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:44.396611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bdc88-e9d6-4e30-ac58-e736eaa7579f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:44.398432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bdc88-e9d6-4e30-ac58-e736eaa7579f service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:44.407782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1010/2039] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:48.078548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9bbd1-fbb5-4492-ab5a-136afefe877a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:48.079974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6ce9bbd1-fbb5-4492-ab5a-136afefe877a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:48.110281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:48.146950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9bbd1-fbb5-4492-ab5a-136afefe877a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:48.152483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:48.153358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:48.153645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:48.153951 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] PATH_INFO: `/v3/users/da1660c6d2544b798e7e688d874baf2a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:48.155143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1660c6d2544b798e7e688d874baf2a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:48.157089 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:48.204277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1660c6d2544b798e7e688d874baf2a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:48.204864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1660c6d2544b798e7e688d874baf2a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:48.205487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1660c6d2544b798e7e688d874baf2a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:48.218896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1660c6d2544b798e7e688d874baf2a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:48.227256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1660c6d2544b798e7e688d874baf2a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:48.241328 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1030/2040] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:48 2022] DELETE /identity/v3/users/da1660c6d2544b798e7e688d874baf2a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:48.246876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7abb31-fadf-4ebe-8ffb-50fd5f891560 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:48.247368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7abb31-fadf-4ebe-8ffb-50fd5f891560 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:48.247481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7abb31-fadf-4ebe-8ffb-50fd5f891560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:48.264550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ca7abb31-fadf-4ebe-8ffb-50fd5f891560 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:48.265941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ca7abb31-fadf-4ebe-8ffb-50fd5f891560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:48.314648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:48.327239 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1011/2041] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:48.337461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:48.339153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:48.398844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920adb63-aae1-4442-8eda-ba795d529c51 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:48.402539 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746c3e5-0015-4a96-94a4-caa0f1204245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:48.403469 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746c3e5-0015-4a96-94a4-caa0f1204245 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:48.403759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746c3e5-0015-4a96-94a4-caa0f1204245 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:48.404034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746c3e5-0015-4a96-94a4-caa0f1204245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:48.414665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746c3e5-0015-4a96-94a4-caa0f1204245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:48.416091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746c3e5-0015-4a96-94a4-caa0f1204245 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:48.430855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1031/2042] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:48.754103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7abb31-fadf-4ebe-8ffb-50fd5f891560 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:48.755753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ca7abb31-fadf-4ebe-8ffb-50fd5f891560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:48.768250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7abb31-fadf-4ebe-8ffb-50fd5f891560 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:48.774047 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:48.774928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:48.775200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:48.775494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] PATH_INFO: `/v3/projects/6104a1a3feed4f109b45489280782944` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:48.776735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6104a1a3feed4f109b45489280782944)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:48.778707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:48.870588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6104a1a3feed4f109b45489280782944', 'request_id': 'req-a42202d5-5d4d-4b34-8914-b715f16be55f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:48.877524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6104a1a3feed4f109b45489280782944', 'request_id': 'req-a42202d5-5d4d-4b34-8914-b715f16be55f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:48.884847 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6104a1a3feed4f109b45489280782944', 'request_id': 'req-a42202d5-5d4d-4b34-8914-b715f16be55f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:48.898538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] The token cache is being invalidate because project 6104a1a3feed4f109b45489280782944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:48.898830 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:48.899449 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:48.901037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1012/2043] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:48 2022] DELETE /identity/v3/projects/6104a1a3feed4f109b45489280782944 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:48.905100 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746c3e5-0015-4a96-94a4-caa0f1204245 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:48.915059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746c3e5-0015-4a96-94a4-caa0f1204245 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:48.959271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:48.973089 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d608542-37f8-4abe-868d-770bd80f8265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:48.974021 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d608542-37f8-4abe-868d-770bd80f8265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:48.974310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d608542-37f8-4abe-868d-770bd80f8265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:48.974586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d608542-37f8-4abe-868d-770bd80f8265 admin admin] PATH_INFO: `/v3/users/54a25c655a654048980dae8d116cd580` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:48.975830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d608542-37f8-4abe-868d-770bd80f8265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a25c655a654048980dae8d116cd580)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:48.977806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d608542-37f8-4abe-868d-770bd80f8265 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:49.031234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6d608542-37f8-4abe-868d-770bd80f8265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a25c655a654048980dae8d116cd580'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.031890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6d608542-37f8-4abe-868d-770bd80f8265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a25c655a654048980dae8d116cd580'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.032527 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6d608542-37f8-4abe-868d-770bd80f8265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a25c655a654048980dae8d116cd580'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.046987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6d608542-37f8-4abe-868d-770bd80f8265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a25c655a654048980dae8d116cd580'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.056909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6d608542-37f8-4abe-868d-770bd80f8265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a25c655a654048980dae8d116cd580'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.061565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:49.064441 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:49.071898 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1032/2044] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:48 2022] DELETE /identity/v3/users/54a25c655a654048980dae8d116cd580 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:49.078826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42202d5-5d4d-4b34-8914-b715f16be55f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:49.146313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:49.147480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.147772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.148052 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] PATH_INFO: `/v3/users/03c9333ca2da43fda0b422641c6226ab` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.149360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03c9333ca2da43fda0b422641c6226ab)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:49.151341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:49.199232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03c9333ca2da43fda0b422641c6226ab'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.206202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03c9333ca2da43fda0b422641c6226ab'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.207301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03c9333ca2da43fda0b422641c6226ab'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.221935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03c9333ca2da43fda0b422641c6226ab'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.236792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03c9333ca2da43fda0b422641c6226ab'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.238521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:49.246254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1013/2045] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:49 2022] DELETE /identity/v3/users/03c9333ca2da43fda0b422641c6226ab => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:49.251066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b08ea6-8ba0-4f27-b9a4-a432c565a190 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.251565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b08ea6-8ba0-4f27-b9a4-a432c565a190 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.251647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b08ea6-8ba0-4f27-b9a4-a432c565a190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.271279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-04b08ea6-8ba0-4f27-b9a4-a432c565a190 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:49.272560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-04b08ea6-8ba0-4f27-b9a4-a432c565a190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:49.339063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1033/2046] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:49.347246 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:49.349224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:49.368310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b08ea6-8ba0-4f27-b9a4-a432c565a190 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:49.370113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-04b08ea6-8ba0-4f27-b9a4-a432c565a190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:49.388289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b08ea6-8ba0-4f27-b9a4-a432c565a190 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:49.393114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:49.394071 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.394352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.394626 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] PATH_INFO: `/v3/projects/b4e24e49a8b74947be8bb801279c2cf3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.396887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e24e49a8b74947be8bb801279c2cf3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:49.399494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:49.432852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3578502-a152-4a5b-aa56-2648ea90c5a5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:49.437346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ea37e-7292-401c-ab2d-13eb45193964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:49.438155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ea37e-7292-401c-ab2d-13eb45193964 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.438512 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ea37e-7292-401c-ab2d-13eb45193964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.438872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ea37e-7292-401c-ab2d-13eb45193964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.449094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ea37e-7292-401c-ab2d-13eb45193964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:49.450569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ea37e-7292-401c-ab2d-13eb45193964 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:49.450946 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e24e49a8b74947be8bb801279c2cf3', 'request_id': 'req-35f7366a-94c0-494d-a5c6-01180b081c6b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.457448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e24e49a8b74947be8bb801279c2cf3', 'request_id': 'req-35f7366a-94c0-494d-a5c6-01180b081c6b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.460526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1014/2047] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:49.465207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e24e49a8b74947be8bb801279c2cf3', 'request_id': 'req-35f7366a-94c0-494d-a5c6-01180b081c6b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.467368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:49.479139 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] The token cache is being invalidate because project b4e24e49a8b74947be8bb801279c2cf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:49.479434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.480100 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-35f7366a-94c0-494d-a5c6-01180b081c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.481746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1034/2048] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:49 2022] DELETE /identity/v3/projects/b4e24e49a8b74947be8bb801279c2cf3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:49.486678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4df2fe-4393-489e-8c27-16c383adc8e0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.486978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4df2fe-4393-489e-8c27-16c383adc8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.487264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4df2fe-4393-489e-8c27-16c383adc8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.505281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5d4df2fe-4393-489e-8c27-16c383adc8e0 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:49.507209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5d4df2fe-4393-489e-8c27-16c383adc8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:49.574908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1015/2049] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:49.581131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5139d0-5f00-4eb7-81c1-bcd5b382bd1c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.581427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5139d0-5f00-4eb7-81c1-bcd5b382bd1c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.581740 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5139d0-5f00-4eb7-81c1-bcd5b382bd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.611242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2f5139d0-5f00-4eb7-81c1-bcd5b382bd1c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:49.612702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2f5139d0-5f00-4eb7-81c1-bcd5b382bd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:49.621683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1035/2050] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:49.625896 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4df2fe-4393-489e-8c27-16c383adc8e0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:49.642518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4df2fe-4393-489e-8c27-16c383adc8e0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:49.647676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c6542-c815-45b7-af51-82872a53d29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:49.648352 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c6542-c815-45b7-af51-82872a53d29c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.648635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c6542-c815-45b7-af51-82872a53d29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.648916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c6542-c815-45b7-af51-82872a53d29c admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.649651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c6542-c815-45b7-af51-82872a53d29c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:49.650074 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c6542-c815-45b7-af51-82872a53d29c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:49.651694 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c6542-c815-45b7-af51-82872a53d29c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:49.665458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1016/2051] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:21:49.670037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5139d0-5f00-4eb7-81c1-bcd5b382bd1c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:49.686010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c6542-c815-45b7-af51-82872a53d29c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:49.686950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5139d0-5f00-4eb7-81c1-bcd5b382bd1c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:49.687843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-863c6542-c815-45b7-af51-82872a53d29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:49.692264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efd13d-1004-4380-a0fc-11e13a8d7905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:49.692940 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efd13d-1004-4380-a0fc-11e13a8d7905 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.693227 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efd13d-1004-4380-a0fc-11e13a8d7905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.693508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efd13d-1004-4380-a0fc-11e13a8d7905 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.694468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efd13d-1004-4380-a0fc-11e13a8d7905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:49.696553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efd13d-1004-4380-a0fc-11e13a8d7905 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:49.702123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c6542-c815-45b7-af51-82872a53d29c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:49.707320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:49.708506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.708653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.708946 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] PATH_INFO: `/v3/users/569a192bd1084b379aa40ceab7fb1c31` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.711496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=569a192bd1084b379aa40ceab7fb1c31)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:49.713434 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:49.729135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1036/2052] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:49 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:49.733517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efd13d-1004-4380-a0fc-11e13a8d7905 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:49.770983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '569a192bd1084b379aa40ceab7fb1c31'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.785474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '569a192bd1084b379aa40ceab7fb1c31'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.786132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '569a192bd1084b379aa40ceab7fb1c31'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.808901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efd13d-1004-4380-a0fc-11e13a8d7905 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:49.810934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '569a192bd1084b379aa40ceab7fb1c31'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.813690 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1889520-c0ed-41cd-958e-fecba808dc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:49.814427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1889520-c0ed-41cd-958e-fecba808dc23 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.814716 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1889520-c0ed-41cd-958e-fecba808dc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.814998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1889520-c0ed-41cd-958e-fecba808dc23 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.815686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1889520-c0ed-41cd-958e-fecba808dc23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:49.817655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1889520-c0ed-41cd-958e-fecba808dc23 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:49.818215 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '569a192bd1084b379aa40ceab7fb1c31'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:49.821378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:49.826727 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1017/2053] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:49 2022] DELETE /identity/v3/users/569a192bd1084b379aa40ceab7fb1c31 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:49.828294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:49.830040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:49.840464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1037/2054] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:49 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:49.843304 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520aeb4-795d-4fe6-806c-803cd32ccba1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.843642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520aeb4-795d-4fe6-806c-803cd32ccba1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.843921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520aeb4-795d-4fe6-806c-803cd32ccba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.862711 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2520aeb4-795d-4fe6-806c-803cd32ccba1 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:49.864376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2520aeb4-795d-4fe6-806c-803cd32ccba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:49.919283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4417a5-bd74-4533-822c-5953e44c2d27 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:49.925945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:49.926985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.927283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.927563 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] PATH_INFO: `/v3/projects/9cd8ebb622b04615ad8c2661bc83ca28` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.929258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cd8ebb622b04615ad8c2661bc83ca28)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:49.931737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:49.958096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1038/2055] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:49.959321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520aeb4-795d-4fe6-806c-803cd32ccba1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:49.961448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2520aeb4-795d-4fe6-806c-803cd32ccba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:49.974310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520aeb4-795d-4fe6-806c-803cd32ccba1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:49.981210 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de76e8d-3695-4c14-8351-7766187d1978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:49.982263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de76e8d-3695-4c14-8351-7766187d1978 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:49.982699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de76e8d-3695-4c14-8351-7766187d1978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:49.983125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de76e8d-3695-4c14-8351-7766187d1978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:49.984023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de76e8d-3695-4c14-8351-7766187d1978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:49.986224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de76e8d-3695-4c14-8351-7766187d1978 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:49.998440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1039/2056] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:49.999447 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de76e8d-3695-4c14-8351-7766187d1978 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:50.003774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9de76e8d-3695-4c14-8351-7766187d1978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.004509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cd8ebb622b04615ad8c2661bc83ca28', 'request_id': 'req-9958195f-e910-459d-b7e8-ed395c9fa40c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.014170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cd8ebb622b04615ad8c2661bc83ca28', 'request_id': 'req-9958195f-e910-459d-b7e8-ed395c9fa40c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.023664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cd8ebb622b04615ad8c2661bc83ca28', 'request_id': 'req-9958195f-e910-459d-b7e8-ed395c9fa40c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.040888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] The token cache is being invalidate because project 9cd8ebb622b04615ad8c2661bc83ca28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:50.041255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.042299 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.044248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1018/2057] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:49 2022] DELETE /identity/v3/projects/9cd8ebb622b04615ad8c2661bc83ca28 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:50.052817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:50.052817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.064113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:50.078240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de76e8d-3695-4c14-8351-7766187d1978 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:50.115553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958195f-e910-459d-b7e8-ed395c9fa40c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:50.120323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d36360-c735-4bb5-84b1-9db81e55cb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:50.121079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d36360-c735-4bb5-84b1-9db81e55cb20 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.121319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d36360-c735-4bb5-84b1-9db81e55cb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.121548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d36360-c735-4bb5-84b1-9db81e55cb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.122303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d36360-c735-4bb5-84b1-9db81e55cb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:50.123899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d36360-c735-4bb5-84b1-9db81e55cb20 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:50.128919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edfc1f-41eb-442f-88d8-c153d0e6ef56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:50.129563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edfc1f-41eb-442f-88d8-c153d0e6ef56 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.129877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edfc1f-41eb-442f-88d8-c153d0e6ef56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.130155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edfc1f-41eb-442f-88d8-c153d0e6ef56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.132552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e3edfc1f-41eb-442f-88d8-c153d0e6ef56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.135031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1019/2058] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:50.135972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d36360-c735-4bb5-84b1-9db81e55cb20 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:50.138196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e6d36360-c735-4bb5-84b1-9db81e55cb20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.144720 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edfc1f-41eb-442f-88d8-c153d0e6ef56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:50.146215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edfc1f-41eb-442f-88d8-c153d0e6ef56 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:50.152096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d36360-c735-4bb5-84b1-9db81e55cb20 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:50.156501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1040/2059] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:50.156741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:50.157848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edfc1f-41eb-442f-88d8-c153d0e6ef56 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:50.158001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.158001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.158182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] PATH_INFO: `/v3/users/8991b28d60ec4bb68dfa7035f53037dc` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.159507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] RBAC: Authorizing `identity:delete_user(user_id=8991b28d60ec4bb68dfa7035f53037dc)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:50.161433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:50.168354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edfc1f-41eb-442f-88d8-c153d0e6ef56 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:50.173959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f0062-632d-454f-826a-aa3e2dc3085c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:50.174611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f0062-632d-454f-826a-aa3e2dc3085c admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.174784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f0062-632d-454f-826a-aa3e2dc3085c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.175048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f0062-632d-454f-826a-aa3e2dc3085c admin admin] PATH_INFO: `/v3/projects/f09afeed1caa455781da41ea97b2426f/users/9ca7b8174734486784b728086b139e94/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.188906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:50.194519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f0062-632d-454f-826a-aa3e2dc3085c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f09afeed1caa455781da41ea97b2426f, user_id=9ca7b8174734486784b728086b139e94, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:50.197216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f0062-632d-454f-826a-aa3e2dc3085c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:50.215386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1041/2060] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:50 2022] PUT /identity/v3/projects/f09afeed1caa455781da41ea97b2426f/users/9ca7b8174734486784b728086b139e94/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:50.221881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8991b28d60ec4bb68dfa7035f53037dc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.229311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5377d6-8a88-4760-a2cc-c26f71c12331 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.229661 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5377d6-8a88-4760-a2cc-c26f71c12331 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.230012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5377d6-8a88-4760-a2cc-c26f71c12331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.231772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8991b28d60ec4bb68dfa7035f53037dc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.232553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8991b28d60ec4bb68dfa7035f53037dc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.251523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8991b28d60ec4bb68dfa7035f53037dc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.259868 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8991b28d60ec4bb68dfa7035f53037dc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.264206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-be5377d6-8a88-4760-a2cc-c26f71c12331 None None] MFA Rules not processed for user `9ca7b8174734486784b728086b139e94`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:50.265643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-be5377d6-8a88-4760-a2cc-c26f71c12331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.266795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1020/2061] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:50 2022] DELETE /identity/v3/users/8991b28d60ec4bb68dfa7035f53037dc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:50.353839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1042/2062] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:50.365188 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:50.367431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.446563 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169bc5d-8808-4885-9282-2a1a4862f3da admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:50.451333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9b8c2-f1fa-4eb3-9554-7b897d19b02a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:50.454490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9b8c2-f1fa-4eb3-9554-7b897d19b02a service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.454490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9b8c2-f1fa-4eb3-9554-7b897d19b02a service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.454490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9b8c2-f1fa-4eb3-9554-7b897d19b02a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.466881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9b8c2-f1fa-4eb3-9554-7b897d19b02a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:50.468437 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9b8c2-f1fa-4eb3-9554-7b897d19b02a service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:50.478800 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1021/2063] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:50.493353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5377d6-8a88-4760-a2cc-c26f71c12331 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:50.495516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-be5377d6-8a88-4760-a2cc-c26f71c12331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.578593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5377d6-8a88-4760-a2cc-c26f71c12331 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:50.583785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:50.584752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.585708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.586495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] PATH_INFO: `/v3/projects/7a02f436f9a34b29b069ca4e5551f8df` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.587718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a02f436f9a34b29b069ca4e5551f8df)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:50.589674 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:50.601194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:50.633737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9b8c2-f1fa-4eb3-9554-7b897d19b02a service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:50.635797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a5b9b8c2-f1fa-4eb3-9554-7b897d19b02a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.660275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a02f436f9a34b29b069ca4e5551f8df', 'request_id': 'req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.669950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a02f436f9a34b29b069ca4e5551f8df', 'request_id': 'req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.674069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9b8c2-f1fa-4eb3-9554-7b897d19b02a service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:50.680129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a02f436f9a34b29b069ca4e5551f8df', 'request_id': 'req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.692433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] The token cache is being invalidate because project 7a02f436f9a34b29b069ca4e5551f8df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:50.692719 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.693651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ac5dfc2e-f29f-4e1b-8a0a-04b865e62966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.696111 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1043/2064] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:50 2022] DELETE /identity/v3/projects/7a02f436f9a34b29b069ca4e5551f8df => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:50.700552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b770c9b-6f4e-4da1-ae2f-5e678ba8c41b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.700846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b770c9b-6f4e-4da1-ae2f-5e678ba8c41b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.701125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b770c9b-6f4e-4da1-ae2f-5e678ba8c41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.726105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7b770c9b-6f4e-4da1-ae2f-5e678ba8c41b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:50.727330 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b770c9b-6f4e-4da1-ae2f-5e678ba8c41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.743033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:50.743655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.744175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.744551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] PATH_INFO: `/v3/projects/b560904a5aca42ee9bdcf3c7a0cc25ed` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.745769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b560904a5aca42ee9bdcf3c7a0cc25ed)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:50.747696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:50.805556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:50.815292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1044/2065] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:50.816487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b560904a5aca42ee9bdcf3c7a0cc25ed', 'request_id': 'req-d525896d-a79b-4d64-863c-a18c909afbba'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.816854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88869157-c712-4f45-be12-ff29f74ea172 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.817162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88869157-c712-4f45-be12-ff29f74ea172 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.817534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88869157-c712-4f45-be12-ff29f74ea172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.825326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b560904a5aca42ee9bdcf3c7a0cc25ed', 'request_id': 'req-d525896d-a79b-4d64-863c-a18c909afbba'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.831917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b560904a5aca42ee9bdcf3c7a0cc25ed', 'request_id': 'req-d525896d-a79b-4d64-863c-a18c909afbba'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.838398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-88869157-c712-4f45-be12-ff29f74ea172 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:50.839997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-88869157-c712-4f45-be12-ff29f74ea172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.849748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] The token cache is being invalidate because project b560904a5aca42ee9bdcf3c7a0cc25ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:50.850318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.851395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d525896d-a79b-4d64-863c-a18c909afbba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:50.853181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1022/2066] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:50 2022] DELETE /identity/v3/projects/b560904a5aca42ee9bdcf3c7a0cc25ed => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:50.859083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca718e-d8ab-4f40-9c78-ed6720b62af0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.859474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca718e-d8ab-4f40-9c78-ed6720b62af0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.859756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca718e-d8ab-4f40-9c78-ed6720b62af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.882602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-aaca718e-d8ab-4f40-9c78-ed6720b62af0 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:50.883826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aaca718e-d8ab-4f40-9c78-ed6720b62af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.910177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1045/2067] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:50.915558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88869157-c712-4f45-be12-ff29f74ea172 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:50.924615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88869157-c712-4f45-be12-ff29f74ea172 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:50.930288 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd8d5a4-3b7f-4281-ab59-040ea9b14311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:50.930938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8d5a4-3b7f-4281-ab59-040ea9b14311 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.931251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8d5a4-3b7f-4281-ab59-040ea9b14311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.931520 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8d5a4-3b7f-4281-ab59-040ea9b14311 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.932246 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd8d5a4-3b7f-4281-ab59-040ea9b14311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:50.932579 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd8d5a4-3b7f-4281-ab59-040ea9b14311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:50.934226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd8d5a4-3b7f-4281-ab59-040ea9b14311 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:50.946269 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1046/2068] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:21:50.946866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1023/2069] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:50.950786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd8d5a4-3b7f-4281-ab59-040ea9b14311 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:50.954285 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62c1a0-4430-4c4e-a315-96e92006a0fe None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.954693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62c1a0-4430-4c4e-a315-96e92006a0fe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.954983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62c1a0-4430-4c4e-a315-96e92006a0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.961122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd8d5a4-3b7f-4281-ab59-040ea9b14311 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:50.965925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08a541-9ced-4dfa-8f1c-71bddf173432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:50.966572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08a541-9ced-4dfa-8f1c-71bddf173432 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:50.966884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08a541-9ced-4dfa-8f1c-71bddf173432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:50.967162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08a541-9ced-4dfa-8f1c-71bddf173432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:50.967852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08a541-9ced-4dfa-8f1c-71bddf173432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:50.969798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08a541-9ced-4dfa-8f1c-71bddf173432 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:50.973256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cb62c1a0-4430-4c4e-a315-96e92006a0fe None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:50.974477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cb62c1a0-4430-4c4e-a315-96e92006a0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:50.983066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1024/2070] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:50.987322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62c1a0-4430-4c4e-a315-96e92006a0fe None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:50.997158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62c1a0-4430-4c4e-a315-96e92006a0fe None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:50.997801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1047/2071] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:50 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:51.002467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08a541-9ced-4dfa-8f1c-71bddf173432 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.007197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:51.064164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9d76a-2197-4e79-927a-ea54f38c8b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.064868 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9d76a-2197-4e79-927a-ea54f38c8b82 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.065156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9d76a-2197-4e79-927a-ea54f38c8b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.065491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9d76a-2197-4e79-927a-ea54f38c8b82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.066230 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9d76a-2197-4e79-927a-ea54f38c8b82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:51.066537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9d76a-2197-4e79-927a-ea54f38c8b82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.069116 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9d76a-2197-4e79-927a-ea54f38c8b82 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.079922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08a541-9ced-4dfa-8f1c-71bddf173432 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.084257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1025/2072] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:21:51.085216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e1505-c7ad-46e5-ab1b-e887aa61e78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.085926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e1505-c7ad-46e5-ab1b-e887aa61e78b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.086217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e1505-c7ad-46e5-ab1b-e887aa61e78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.086499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e1505-c7ad-46e5-ab1b-e887aa61e78b admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.087192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e1505-c7ad-46e5-ab1b-e887aa61e78b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.088926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9d76a-2197-4e79-927a-ea54f38c8b82 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.089478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e1505-c7ad-46e5-ab1b-e887aa61e78b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.104507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9d76a-2197-4e79-927a-ea54f38c8b82 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.110996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086f8a3-87a1-4a77-b930-b37a40a4dfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.111794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086f8a3-87a1-4a77-b930-b37a40a4dfd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.112122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086f8a3-87a1-4a77-b930-b37a40a4dfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.112471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086f8a3-87a1-4a77-b930-b37a40a4dfd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.113251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086f8a3-87a1-4a77-b930-b37a40a4dfd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.115345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086f8a3-87a1-4a77-b930-b37a40a4dfd2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.117899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1048/2073] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:51 2022] POST /identity/v3/users => generated 303 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:51.119153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e1505-c7ad-46e5-ab1b-e887aa61e78b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.121667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-eb1e1505-c7ad-46e5-ab1b-e887aa61e78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:51.136006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e1505-c7ad-46e5-ab1b-e887aa61e78b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.141435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.142333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.142621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.142897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] PATH_INFO: `/v3/users/a7b165eab1ed4da2ad3b7d31459ce40c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.144105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b165eab1ed4da2ad3b7d31459ce40c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.146122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.149877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1026/2074] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:51 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:51.150779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086f8a3-87a1-4a77-b930-b37a40a4dfd2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.206646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:51.212160 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b165eab1ed4da2ad3b7d31459ce40c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:51.212914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b165eab1ed4da2ad3b7d31459ce40c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:51.213701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b165eab1ed4da2ad3b7d31459ce40c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:51.229500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b165eab1ed4da2ad3b7d31459ce40c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:51.236458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086f8a3-87a1-4a77-b930-b37a40a4dfd2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.239949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b165eab1ed4da2ad3b7d31459ce40c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:51.251321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca49b4-4a75-474c-b5d1-53acfceeadd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.252004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca49b4-4a75-474c-b5d1-53acfceeadd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.252314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca49b4-4a75-474c-b5d1-53acfceeadd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.252574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca49b4-4a75-474c-b5d1-53acfceeadd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.253292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca49b4-4a75-474c-b5d1-53acfceeadd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.254995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca49b4-4a75-474c-b5d1-53acfceeadd1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.258615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1049/2075] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:51 2022] DELETE /identity/v3/users/a7b165eab1ed4da2ad3b7d31459ce40c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:51.262303 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.263663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:51.266768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1027/2076] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:51.268030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549fc92-19fe-434f-97d2-e0ae0072f7ca None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.268330 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549fc92-19fe-434f-97d2-e0ae0072f7ca None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.268611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549fc92-19fe-434f-97d2-e0ae0072f7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.285987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5549fc92-19fe-434f-97d2-e0ae0072f7ca None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:51.287189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5549fc92-19fe-434f-97d2-e0ae0072f7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:51.333811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195d2d8-1ed3-49eb-8976-1693ed48e0f3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.339192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129014b-2bf7-4f5f-ab11-55310ab1526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.339855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129014b-2bf7-4f5f-ab11-55310ab1526d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.340136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129014b-2bf7-4f5f-ab11-55310ab1526d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.340408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129014b-2bf7-4f5f-ab11-55310ab1526d admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.341086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129014b-2bf7-4f5f-ab11-55310ab1526d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.343074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129014b-2bf7-4f5f-ab11-55310ab1526d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.350719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1028/2077] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:51.351994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549fc92-19fe-434f-97d2-e0ae0072f7ca None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.352862 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:51.354409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5549fc92-19fe-434f-97d2-e0ae0072f7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:51.367215 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549fc92-19fe-434f-97d2-e0ae0072f7ca None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.369596 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1050/2078] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:51 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:51.370618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129014b-2bf7-4f5f-ab11-55310ab1526d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.372401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e129014b-2bf7-4f5f-ab11-55310ab1526d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:51.375288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8b1d7-8a07-4d22-8935-3036a98d3223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.375965 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8b1d7-8a07-4d22-8935-3036a98d3223 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.376252 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8b1d7-8a07-4d22-8935-3036a98d3223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.376534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8b1d7-8a07-4d22-8935-3036a98d3223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.377323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8b1d7-8a07-4d22-8935-3036a98d3223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.378989 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8b1d7-8a07-4d22-8935-3036a98d3223 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.390052 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1029/2079] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:51.390985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8b1d7-8a07-4d22-8935-3036a98d3223 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.401402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8b1d7-8a07-4d22-8935-3036a98d3223 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.406149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbd71f-9813-4bd8-bdc5-15243e8d3c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.406794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbd71f-9813-4bd8-bdc5-15243e8d3c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.407079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbd71f-9813-4bd8-bdc5-15243e8d3c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.407365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbd71f-9813-4bd8-bdc5-15243e8d3c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.408094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbd71f-9813-4bd8-bdc5-15243e8d3c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.409662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbd71f-9813-4bd8-bdc5-15243e8d3c9b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.420110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1030/2080] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:51.420870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbd71f-9813-4bd8-bdc5-15243e8d3c9b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.435087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbd71f-9813-4bd8-bdc5-15243e8d3c9b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.444498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454b01e-6099-4b5b-a9df-ee626510ffdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.444780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129014b-2bf7-4f5f-ab11-55310ab1526d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.445522 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454b01e-6099-4b5b-a9df-ee626510ffdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.445780 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454b01e-6099-4b5b-a9df-ee626510ffdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.446065 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454b01e-6099-4b5b-a9df-ee626510ffdc admin admin] PATH_INFO: `/v3/projects/40edd1b3602f4cc1b21a959fea954f88/users/0d5f1e09bcd048a99b8a5a07bffd545d/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.448898 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba54fd4-190e-4a73-be21-a299c866aa0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.449534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba54fd4-190e-4a73-be21-a299c866aa0e service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.449854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba54fd4-190e-4a73-be21-a299c866aa0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.450137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba54fd4-190e-4a73-be21-a299c866aa0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.459956 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba54fd4-190e-4a73-be21-a299c866aa0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.461502 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454b01e-6099-4b5b-a9df-ee626510ffdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=40edd1b3602f4cc1b21a959fea954f88, user_id=0d5f1e09bcd048a99b8a5a07bffd545d, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.461636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba54fd4-190e-4a73-be21-a299c866aa0e service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.464258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454b01e-6099-4b5b-a9df-ee626510ffdc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.471247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1051/2081] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:51.474609 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba54fd4-190e-4a73-be21-a299c866aa0e service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.479198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1031/2082] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:51 2022] PUT /identity/v3/projects/40edd1b3602f4cc1b21a959fea954f88/users/0d5f1e09bcd048a99b8a5a07bffd545d/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:51.489578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94691ab-6a3c-4dd0-a367-aa09a48335f8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.489938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94691ab-6a3c-4dd0-a367-aa09a48335f8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.490230 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94691ab-6a3c-4dd0-a367-aa09a48335f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.497330 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba54fd4-190e-4a73-be21-a299c866aa0e service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.519402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a94691ab-6a3c-4dd0-a367-aa09a48335f8 None None] MFA Rules not processed for user `0d5f1e09bcd048a99b8a5a07bffd545d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:51.520724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a94691ab-6a3c-4dd0-a367-aa09a48335f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:51.570011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef180e8b-5d9c-4a8b-baed-894d94781cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.570816 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef180e8b-5d9c-4a8b-baed-894d94781cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.571098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef180e8b-5d9c-4a8b-baed-894d94781cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.571372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef180e8b-5d9c-4a8b-baed-894d94781cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.572201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef180e8b-5d9c-4a8b-baed-894d94781cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.574718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef180e8b-5d9c-4a8b-baed-894d94781cf0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.587243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1052/2083] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:51.591368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef180e8b-5d9c-4a8b-baed-894d94781cf0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.599222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1032/2084] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:51 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:51.600663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94691ab-6a3c-4dd0-a367-aa09a48335f8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.604848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef180e8b-5d9c-4a8b-baed-894d94781cf0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.609643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0036a4e-de8a-4d8b-b714-bbe764901460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.611953 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94691ab-6a3c-4dd0-a367-aa09a48335f8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.613635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0036a4e-de8a-4d8b-b714-bbe764901460 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.613957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0036a4e-de8a-4d8b-b714-bbe764901460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.614242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0036a4e-de8a-4d8b-b714-bbe764901460 admin admin] PATH_INFO: `/v3/projects/24ea377e9c51407ca742c5ce93e6c274/users/baf5566de2654de18e0fa38756fb72e2/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.617379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25db06f-12a2-4f02-a081-751778c65383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.618181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25db06f-12a2-4f02-a081-751778c65383 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.618466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25db06f-12a2-4f02-a081-751778c65383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.618743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25db06f-12a2-4f02-a081-751778c65383 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.619433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25db06f-12a2-4f02-a081-751778c65383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.621394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25db06f-12a2-4f02-a081-751778c65383 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.633746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0036a4e-de8a-4d8b-b714-bbe764901460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24ea377e9c51407ca742c5ce93e6c274, user_id=baf5566de2654de18e0fa38756fb72e2, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.638032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:51.638406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0036a4e-de8a-4d8b-b714-bbe764901460 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.656909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1053/2085] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:51 2022] PUT /identity/v3/projects/24ea377e9c51407ca742c5ce93e6c274/users/baf5566de2654de18e0fa38756fb72e2/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:51.657942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1033/2086] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:51 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:51.659194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0036a4e-de8a-4d8b-b714-bbe764901460 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.669401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be98a35-8f01-4199-aaa4-d021f7466b49 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.669695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be98a35-8f01-4199-aaa4-d021f7466b49 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.670016 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be98a35-8f01-4199-aaa4-d021f7466b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.706138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8be98a35-8f01-4199-aaa4-d021f7466b49 None None] MFA Rules not processed for user `baf5566de2654de18e0fa38756fb72e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:51.707484 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8be98a35-8f01-4199-aaa4-d021f7466b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:51.738706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0036a4e-de8a-4d8b-b714-bbe764901460 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.755955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:51.789799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1034/2087] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:51.790429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be98a35-8f01-4199-aaa4-d021f7466b49 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.792143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8be98a35-8f01-4199-aaa4-d021f7466b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:51.810676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b959c-b96a-42a2-8871-ef305f5318e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.811413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b959c-b96a-42a2-8871-ef305f5318e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.811709 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b959c-b96a-42a2-8871-ef305f5318e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.811983 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b959c-b96a-42a2-8871-ef305f5318e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.812996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b959c-b96a-42a2-8871-ef305f5318e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.814947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b959c-b96a-42a2-8871-ef305f5318e8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.838502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1054/2088] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:51 2022] POST /identity/v3/users => generated 309 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:51.839482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b959c-b96a-42a2-8871-ef305f5318e8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.841428 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fc1b959c-b96a-42a2-8871-ef305f5318e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:51.878427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be98a35-8f01-4199-aaa4-d021f7466b49 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.889618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.890863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.891553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.891683 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] PATH_INFO: `/v3/projects/079d5c88293d48e682e922fd6b2faade` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.893369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=079d5c88293d48e682e922fd6b2faade)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.895538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.916692 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b959c-b96a-42a2-8871-ef305f5318e8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.920401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb06e8-64de-4ea9-8f04-d1ac2e6c3328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:51.921050 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb06e8-64de-4ea9-8f04-d1ac2e6c3328 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:51.921508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb06e8-64de-4ea9-8f04-d1ac2e6c3328 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:51.921619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb06e8-64de-4ea9-8f04-d1ac2e6c3328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:51.932353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb06e8-64de-4ea9-8f04-d1ac2e6c3328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:51.934314 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb06e8-64de-4ea9-8f04-d1ac2e6c3328 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:51.949687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1055/2089] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:51.950578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb06e8-64de-4ea9-8f04-d1ac2e6c3328 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:51.960716 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079d5c88293d48e682e922fd6b2faade', 'request_id': 'req-b4a12136-bcab-4855-bf0e-89d6c15c68e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:51.965190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb06e8-64de-4ea9-8f04-d1ac2e6c3328 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:51.967010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079d5c88293d48e682e922fd6b2faade', 'request_id': 'req-b4a12136-bcab-4855-bf0e-89d6c15c68e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:51.975019 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079d5c88293d48e682e922fd6b2faade', 'request_id': 'req-b4a12136-bcab-4855-bf0e-89d6c15c68e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:51.997551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] The token cache is being invalidate because project 079d5c88293d48e682e922fd6b2faade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:51.997883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:51.999148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b4a12136-bcab-4855-bf0e-89d6c15c68e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:52.001124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1035/2090] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:51 2022] DELETE /identity/v3/projects/079d5c88293d48e682e922fd6b2faade => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:52.010370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5704e-706c-43e1-983f-2c62ba79b6bc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.010746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5704e-706c-43e1-983f-2c62ba79b6bc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.011092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5704e-706c-43e1-983f-2c62ba79b6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.038666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-30b5704e-706c-43e1-983f-2c62ba79b6bc None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:52.040720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-30b5704e-706c-43e1-983f-2c62ba79b6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:52.052724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:52.090297 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6f5a8-9868-4f6b-a591-aecfb97235c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.090932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6f5a8-9868-4f6b-a591-aecfb97235c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.093064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6f5a8-9868-4f6b-a591-aecfb97235c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.093565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6f5a8-9868-4f6b-a591-aecfb97235c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.094479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6f5a8-9868-4f6b-a591-aecfb97235c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.096119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6f5a8-9868-4f6b-a591-aecfb97235c5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.111319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1056/2091] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:52.116405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6f5a8-9868-4f6b-a591-aecfb97235c5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:52.119162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-90c6f5a8-9868-4f6b-a591-aecfb97235c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:52.135251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6f5a8-9868-4f6b-a591-aecfb97235c5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:52.140267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e80ea-9ed8-4934-bf10-6f42b8774f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.141283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e80ea-9ed8-4934-bf10-6f42b8774f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.141578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e80ea-9ed8-4934-bf10-6f42b8774f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.141885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e80ea-9ed8-4934-bf10-6f42b8774f82 admin admin] PATH_INFO: `/v3/projects/476c91c8839a43d99aa9db2a0f53b0eb/users/579ed3e31e98444791ed37fda997a215/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.148002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1036/2092] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:52.156173 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ee18a-73ef-4a8e-ae17-6e887b42ff74 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.156551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ee18a-73ef-4a8e-ae17-6e887b42ff74 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.156840 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ee18a-73ef-4a8e-ae17-6e887b42ff74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.163761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e80ea-9ed8-4934-bf10-6f42b8774f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=476c91c8839a43d99aa9db2a0f53b0eb, user_id=579ed3e31e98444791ed37fda997a215, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.167835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e80ea-9ed8-4934-bf10-6f42b8774f82 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.180678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-068ee18a-73ef-4a8e-ae17-6e887b42ff74 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:52.182011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-068ee18a-73ef-4a8e-ae17-6e887b42ff74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:52.184977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1057/2093] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:52 2022] PUT /identity/v3/projects/476c91c8839a43d99aa9db2a0f53b0eb/users/579ed3e31e98444791ed37fda997a215/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:52.190913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e80ea-9ed8-4934-bf10-6f42b8774f82 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:52.221511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:52.267418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1037/2094] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:52.274775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ee18a-73ef-4a8e-ae17-6e887b42ff74 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:52.286122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ee18a-73ef-4a8e-ae17-6e887b42ff74 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:52.290340 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e80ea-9ed8-4934-bf10-6f42b8774f82 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:52.303118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e2933-05c7-4217-8fad-6d0dd8e749ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.303844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e2933-05c7-4217-8fad-6d0dd8e749ba admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.304211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e2933-05c7-4217-8fad-6d0dd8e749ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.304548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e2933-05c7-4217-8fad-6d0dd8e749ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.305445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e2933-05c7-4217-8fad-6d0dd8e749ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.309496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e2933-05c7-4217-8fad-6d0dd8e749ba admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.311799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc98ec6-6c34-48b0-be1c-ae7c02bd2667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.312493 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98ec6-6c34-48b0-be1c-ae7c02bd2667 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.312781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98ec6-6c34-48b0-be1c-ae7c02bd2667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.313060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98ec6-6c34-48b0-be1c-ae7c02bd2667 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.314267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc98ec6-6c34-48b0-be1c-ae7c02bd2667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:52.314583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc98ec6-6c34-48b0-be1c-ae7c02bd2667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.316318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc98ec6-6c34-48b0-be1c-ae7c02bd2667 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.327276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1058/2095] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:52.332040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1038/2096] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:21:52.334476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e2933-05c7-4217-8fad-6d0dd8e749ba admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:52.337327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc98ec6-6c34-48b0-be1c-ae7c02bd2667 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:52.349397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e2933-05c7-4217-8fad-6d0dd8e749ba admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:52.357220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc98ec6-6c34-48b0-be1c-ae7c02bd2667 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:52.358469 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566142e-5e48-4447-bca7-c8bb9396c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.360062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566142e-5e48-4447-bca7-c8bb9396c905 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.360450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566142e-5e48-4447-bca7-c8bb9396c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.360869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566142e-5e48-4447-bca7-c8bb9396c905 admin admin] PATH_INFO: `/v3/projects/476c91c8839a43d99aa9db2a0f53b0eb/users/579ed3e31e98444791ed37fda997a215/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.363849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566142e-5e48-4447-bca7-c8bb9396c905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=476c91c8839a43d99aa9db2a0f53b0eb, user_id=579ed3e31e98444791ed37fda997a215, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.364457 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f3eb5-343c-4cc7-b343-3ae9f1f42d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.365422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f3eb5-343c-4cc7-b343-3ae9f1f42d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.365933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f3eb5-343c-4cc7-b343-3ae9f1f42d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.366367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f3eb5-343c-4cc7-b343-3ae9f1f42d74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.367363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f3eb5-343c-4cc7-b343-3ae9f1f42d74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.369966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566142e-5e48-4447-bca7-c8bb9396c905 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.370640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f3eb5-343c-4cc7-b343-3ae9f1f42d74 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.386411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1059/2097] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:52 2022] PUT /identity/v3/projects/476c91c8839a43d99aa9db2a0f53b0eb/users/579ed3e31e98444791ed37fda997a215/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:52.402770 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef3e0d-9801-410c-97cd-70ee3143863b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.403523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef3e0d-9801-410c-97cd-70ee3143863b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.403811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef3e0d-9801-410c-97cd-70ee3143863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.426847 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1039/2098] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:52 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:52.431382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f3eb5-343c-4cc7-b343-3ae9f1f42d74 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:52.449316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d2ef3e0d-9801-410c-97cd-70ee3143863b None None] MFA Rules not processed for user `579ed3e31e98444791ed37fda997a215`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:52.450732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d2ef3e0d-9801-410c-97cd-70ee3143863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:52.517948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f3eb5-343c-4cc7-b343-3ae9f1f42d74 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:52.525925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab2e4c-cae1-41d6-a375-ad61590d4bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.526610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab2e4c-cae1-41d6-a375-ad61590d4bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.526888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab2e4c-cae1-41d6-a375-ad61590d4bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.527157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab2e4c-cae1-41d6-a375-ad61590d4bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.527852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab2e4c-cae1-41d6-a375-ad61590d4bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.529816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab2e4c-cae1-41d6-a375-ad61590d4bd1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.542569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1060/2099] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:52 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:52.556336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef3e0d-9801-410c-97cd-70ee3143863b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:52.557932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d2ef3e0d-9801-410c-97cd-70ee3143863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:52.565637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1040/2100] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:52 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:52.566732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab2e4c-cae1-41d6-a375-ad61590d4bd1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:52.577691 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab2e4c-cae1-41d6-a375-ad61590d4bd1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:52.583703 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302b32d-3e4d-4518-bb3c-8b24b289cc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.584805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b32d-3e4d-4518-bb3c-8b24b289cc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.585102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b32d-3e4d-4518-bb3c-8b24b289cc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.585623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b32d-3e4d-4518-bb3c-8b24b289cc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.586711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302b32d-3e4d-4518-bb3c-8b24b289cc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.588830 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302b32d-3e4d-4518-bb3c-8b24b289cc3d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.594790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:52.609375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1041/2101] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:21:52.610590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302b32d-3e4d-4518-bb3c-8b24b289cc3d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:52.621556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302b32d-3e4d-4518-bb3c-8b24b289cc3d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:52.625014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef3e0d-9801-410c-97cd-70ee3143863b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:52.626364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789d614-7e47-4d5b-8f95-b384ba17ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.627036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789d614-7e47-4d5b-8f95-b384ba17ace5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.627322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789d614-7e47-4d5b-8f95-b384ba17ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.627656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789d614-7e47-4d5b-8f95-b384ba17ace5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.628318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789d614-7e47-4d5b-8f95-b384ba17ace5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.628849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b368e-a6e4-4fed-a209-7bfa66c527d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.629496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b368e-a6e4-4fed-a209-7bfa66c527d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.629808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b368e-a6e4-4fed-a209-7bfa66c527d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.629963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789d614-7e47-4d5b-8f95-b384ba17ace5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.630090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b368e-a6e4-4fed-a209-7bfa66c527d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.642337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b368e-a6e4-4fed-a209-7bfa66c527d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.648481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1042/2102] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:21:52.648668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b368e-a6e4-4fed-a209-7bfa66c527d6 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.649411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789d614-7e47-4d5b-8f95-b384ba17ace5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:52.658768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1061/2103] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:52 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:21:52.659372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789d614-7e47-4d5b-8f95-b384ba17ace5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:52.665221 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de346bfb-9259-47e4-a3ee-3e0948a5c5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.666137 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de346bfb-9259-47e4-a3ee-3e0948a5c5bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.666423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de346bfb-9259-47e4-a3ee-3e0948a5c5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.666698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de346bfb-9259-47e4-a3ee-3e0948a5c5bd admin admin] PATH_INFO: `/v3/projects/a40431f830294ea0a6395be072560dea/users/c935f035c757482ca912a8c4b77714a0/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.683070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de346bfb-9259-47e4-a3ee-3e0948a5c5bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a40431f830294ea0a6395be072560dea, user_id=c935f035c757482ca912a8c4b77714a0, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.685910 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de346bfb-9259-47e4-a3ee-3e0948a5c5bd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.709940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1043/2104] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:52 2022] PUT /identity/v3/projects/a40431f830294ea0a6395be072560dea/users/c935f035c757482ca912a8c4b77714a0/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:52.716338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a588504-176f-478a-bc27-f11b0e3896eb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.716733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a588504-176f-478a-bc27-f11b0e3896eb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.717112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a588504-176f-478a-bc27-f11b0e3896eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.740344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2a588504-176f-478a-bc27-f11b0e3896eb None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:52.740344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2a588504-176f-478a-bc27-f11b0e3896eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:52.770608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:52.817053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1062/2105] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:52.826922 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de346bfb-9259-47e4-a3ee-3e0948a5c5bd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:52.828636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-de346bfb-9259-47e4-a3ee-3e0948a5c5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:52.886357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de346bfb-9259-47e4-a3ee-3e0948a5c5bd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:52.890044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9892cd9-be16-44d4-857b-d883afc193a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:52.890682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9892cd9-be16-44d4-857b-d883afc193a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:52.890965 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9892cd9-be16-44d4-857b-d883afc193a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:52.891263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9892cd9-be16-44d4-857b-d883afc193a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:52.900973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9892cd9-be16-44d4-857b-d883afc193a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:52.902447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9892cd9-be16-44d4-857b-d883afc193a9 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:52.911659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1044/2106] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:55.309619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a588504-176f-478a-bc27-f11b0e3896eb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:55.329484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a588504-176f-478a-bc27-f11b0e3896eb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:55.341318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:55.342694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:55.342984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:55.343440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] PATH_INFO: `/v3/users/0d5f1e09bcd048a99b8a5a07bffd545d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:55.344764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d5f1e09bcd048a99b8a5a07bffd545d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:55.346781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:55.445918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d5f1e09bcd048a99b8a5a07bffd545d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:55.446682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d5f1e09bcd048a99b8a5a07bffd545d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:55.448063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d5f1e09bcd048a99b8a5a07bffd545d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:55.472320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d5f1e09bcd048a99b8a5a07bffd545d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:55.487722 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d5f1e09bcd048a99b8a5a07bffd545d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:55.511598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1063/2107] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:55 2022] DELETE /identity/v3/users/0d5f1e09bcd048a99b8a5a07bffd545d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:55.517159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4343e1-f228-481c-b31f-013f12996cf1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:55.517603 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4343e1-f228-481c-b31f-013f12996cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:55.517740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4343e1-f228-481c-b31f-013f12996cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:55.538705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1f4343e1-f228-481c-b31f-013f12996cf1 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:55.540590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1f4343e1-f228-481c-b31f-013f12996cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:55.641618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1045/2108] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:55.652684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:55.654439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:55.731999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a5876-536d-471e-8980-e1271b2fa5c6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:55.737478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c6021-ca65-4b2c-97e4-e0155a8c2b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:55.738182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c6021-ca65-4b2c-97e4-e0155a8c2b66 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:55.738469 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c6021-ca65-4b2c-97e4-e0155a8c2b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:55.738798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c6021-ca65-4b2c-97e4-e0155a8c2b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:55.754034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c6021-ca65-4b2c-97e4-e0155a8c2b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:55.755542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c6021-ca65-4b2c-97e4-e0155a8c2b66 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:55.769348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1064/2109] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:56.101537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4343e1-f228-481c-b31f-013f12996cf1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:56.103243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1f4343e1-f228-481c-b31f-013f12996cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:56.126116 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4343e1-f228-481c-b31f-013f12996cf1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:56.143424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:56.144504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:56.145083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:56.145195 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] PATH_INFO: `/v3/projects/40edd1b3602f4cc1b21a959fea954f88` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:56.146495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=40edd1b3602f4cc1b21a959fea954f88)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:56.148939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:56.204790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40edd1b3602f4cc1b21a959fea954f88', 'request_id': 'req-690d750b-3a8c-4eec-8e2c-3d682419d7ae'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:56.212972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40edd1b3602f4cc1b21a959fea954f88', 'request_id': 'req-690d750b-3a8c-4eec-8e2c-3d682419d7ae'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:56.220728 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40edd1b3602f4cc1b21a959fea954f88', 'request_id': 'req-690d750b-3a8c-4eec-8e2c-3d682419d7ae'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:56.223326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:56.250009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] The token cache is being invalidate because project 40edd1b3602f4cc1b21a959fea954f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:56.250415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:56.251406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:56.259638 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c6021-ca65-4b2c-97e4-e0155a8c2b66 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:56.260160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1046/2110] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:56 2022] DELETE /identity/v3/projects/40edd1b3602f4cc1b21a959fea954f88 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:56.274381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c6021-ca65-4b2c-97e4-e0155a8c2b66 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:56.364756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:56.389439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c352c-bd1c-4a9c-94a6-0ccbe518b8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:56.390797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c352c-bd1c-4a9c-94a6-0ccbe518b8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:56.391272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c352c-bd1c-4a9c-94a6-0ccbe518b8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:56.391817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c352c-bd1c-4a9c-94a6-0ccbe518b8b5 admin admin] PATH_INFO: `/v3/users/579ed3e31e98444791ed37fda997a215` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:56.402209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c352c-bd1c-4a9c-94a6-0ccbe518b8b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=579ed3e31e98444791ed37fda997a215)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:56.402209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c352c-bd1c-4a9c-94a6-0ccbe518b8b5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:56.518441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-600c352c-bd1c-4a9c-94a6-0ccbe518b8b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '579ed3e31e98444791ed37fda997a215'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:56.519370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-600c352c-bd1c-4a9c-94a6-0ccbe518b8b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '579ed3e31e98444791ed37fda997a215'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:56.520141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-600c352c-bd1c-4a9c-94a6-0ccbe518b8b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '579ed3e31e98444791ed37fda997a215'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:56.547936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-600c352c-bd1c-4a9c-94a6-0ccbe518b8b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '579ed3e31e98444791ed37fda997a215'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:56.560916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-600c352c-bd1c-4a9c-94a6-0ccbe518b8b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '579ed3e31e98444791ed37fda997a215'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:56.582565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1065/2111] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:21:56 2022] DELETE /identity/v3/users/579ed3e31e98444791ed37fda997a215 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:57.087508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:57.090822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:57.130575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d750b-3a8c-4eec-8e2c-3d682419d7ae admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:57.221622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:57.222696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:57.223568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:57.223568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] PATH_INFO: `/v3/projects/476c91c8839a43d99aa9db2a0f53b0eb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:57.235695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=476c91c8839a43d99aa9db2a0f53b0eb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:57.239261 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:57.327658 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '476c91c8839a43d99aa9db2a0f53b0eb', 'request_id': 'req-f199524f-7b3d-4714-baf1-528a54e1c42a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:57.337123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '476c91c8839a43d99aa9db2a0f53b0eb', 'request_id': 'req-f199524f-7b3d-4714-baf1-528a54e1c42a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:57.347842 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '476c91c8839a43d99aa9db2a0f53b0eb', 'request_id': 'req-f199524f-7b3d-4714-baf1-528a54e1c42a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:57.366814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] The token cache is being invalidate because project 476c91c8839a43d99aa9db2a0f53b0eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:21:57.367870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:57.369646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f199524f-7b3d-4714-baf1-528a54e1c42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:21:57.372557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1047/2112] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:21:57 2022] DELETE /identity/v3/projects/476c91c8839a43d99aa9db2a0f53b0eb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:21:57.380592 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ac0d1-d23f-4261-9ec1-d8ef33d042b8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:57.381229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ac0d1-d23f-4261-9ec1-d8ef33d042b8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:57.381229 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ac0d1-d23f-4261-9ec1-d8ef33d042b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:57.436769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cb7ac0d1-d23f-4261-9ec1-d8ef33d042b8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:57.437975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cb7ac0d1-d23f-4261-9ec1-d8ef33d042b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:57.533785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1066/2113] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:57.541871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491528-c31f-4d4d-9452-471dd0a56b6e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:57.542397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491528-c31f-4d4d-9452-471dd0a56b6e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:57.542950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491528-c31f-4d4d-9452-471dd0a56b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:57.576189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-89491528-c31f-4d4d-9452-471dd0a56b6e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:57.579122 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-89491528-c31f-4d4d-9452-471dd0a56b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:57.595513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ac0d1-d23f-4261-9ec1-d8ef33d042b8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:57.600644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1048/2114] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:57.606839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ac0d1-d23f-4261-9ec1-d8ef33d042b8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:57.612294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebbfce-8978-4ae7-a698-4358f526296c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:57.613160 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebbfce-8978-4ae7-a698-4358f526296c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:57.613459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebbfce-8978-4ae7-a698-4358f526296c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:57.613766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebbfce-8978-4ae7-a698-4358f526296c admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:57.614514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebbfce-8978-4ae7-a698-4358f526296c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:21:57.614824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebbfce-8978-4ae7-a698-4358f526296c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:57.617086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebbfce-8978-4ae7-a698-4358f526296c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:57.634454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1067/2115] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:21:57.638883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89491528-c31f-4d4d-9452-471dd0a56b6e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:57.646016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a821fa-2ed9-4f26-a85c-ccbd8e6151d3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:57.646451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a821fa-2ed9-4f26-a85c-ccbd8e6151d3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:57.646628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a821fa-2ed9-4f26-a85c-ccbd8e6151d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:57.652433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89491528-c31f-4d4d-9452-471dd0a56b6e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:57.662013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329e5fc-c7ac-4fc5-abbb-593a6e47e8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:57.663238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e5fc-c7ac-4fc5-abbb-593a6e47e8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:57.663575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e5fc-c7ac-4fc5-abbb-593a6e47e8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:57.663947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e5fc-c7ac-4fc5-abbb-593a6e47e8e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:57.664717 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3329e5fc-c7ac-4fc5-abbb-593a6e47e8e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:57.666765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3329e5fc-c7ac-4fc5-abbb-593a6e47e8e8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:57.688219 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-48a821fa-2ed9-4f26-a85c-ccbd8e6151d3 None None] MFA Rules not processed for user `c935f035c757482ca912a8c4b77714a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:57.690032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-48a821fa-2ed9-4f26-a85c-ccbd8e6151d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:57.704391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1049/2116] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:57 2022] POST /identity/v3/projects => generated 403 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:57.708743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329e5fc-c7ac-4fc5-abbb-593a6e47e8e8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:57.780010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:57.788973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:57.802365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329e5fc-c7ac-4fc5-abbb-593a6e47e8e8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:57.810648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91b202-4fe7-49ad-b225-a7fccf8a284f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:57.811803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91b202-4fe7-49ad-b225-a7fccf8a284f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:57.812157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91b202-4fe7-49ad-b225-a7fccf8a284f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:57.812509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91b202-4fe7-49ad-b225-a7fccf8a284f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:57.813286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91b202-4fe7-49ad-b225-a7fccf8a284f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:57.815412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91b202-4fe7-49ad-b225-a7fccf8a284f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:57.832496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1068/2117] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:57.837067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a821fa-2ed9-4f26-a85c-ccbd8e6151d3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:57.839258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-48a821fa-2ed9-4f26-a85c-ccbd8e6151d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:57.842420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1050/2118] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:57 2022] POST /identity/v3/users => generated 302 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:21:57.843554 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91b202-4fe7-49ad-b225-a7fccf8a284f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:57.855903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91b202-4fe7-49ad-b225-a7fccf8a284f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:57.856916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a821fa-2ed9-4f26-a85c-ccbd8e6151d3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:57.861069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b3858-9248-436b-9b38-67b6bac12bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:57.861911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b3858-9248-436b-9b38-67b6bac12bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:57.862290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b3858-9248-436b-9b38-67b6bac12bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:57.862403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ab4c9-c7f4-48cf-b631-18e9942db87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:57.862649 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b3858-9248-436b-9b38-67b6bac12bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:57.862971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ab4c9-c7f4-48cf-b631-18e9942db87f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:57.863260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ab4c9-c7f4-48cf-b631-18e9942db87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:57.863537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ab4c9-c7f4-48cf-b631-18e9942db87f admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:57.863646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b3858-9248-436b-9b38-67b6bac12bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:57.864232 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ab4c9-c7f4-48cf-b631-18e9942db87f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:57.865651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b3858-9248-436b-9b38-67b6bac12bcd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:57.867509 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ab4c9-c7f4-48cf-b631-18e9942db87f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:57.880914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1051/2119] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:57.883402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b3858-9248-436b-9b38-67b6bac12bcd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:57.899027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b3858-9248-436b-9b38-67b6bac12bcd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:57.908854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b39366-0fbc-4aca-b140-663d31a646a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:57.909544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b39366-0fbc-4aca-b140-663d31a646a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:57.909871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b39366-0fbc-4aca-b140-663d31a646a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:57.910149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b39366-0fbc-4aca-b140-663d31a646a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:57.910983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b39366-0fbc-4aca-b140-663d31a646a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:57.911184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1069/2120] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:57 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:57.912226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ab4c9-c7f4-48cf-b631-18e9942db87f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:57.913146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b39366-0fbc-4aca-b140-663d31a646a2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:57.925765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1052/2121] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:57.929979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b39366-0fbc-4aca-b140-663d31a646a2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:57.945516 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b39366-0fbc-4aca-b140-663d31a646a2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:57.988306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ab4c9-c7f4-48cf-b631-18e9942db87f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:58.000954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d862bd-9678-4b0d-94fb-2acec9b918ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:58.001640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d862bd-9678-4b0d-94fb-2acec9b918ed admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:58.001970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d862bd-9678-4b0d-94fb-2acec9b918ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:58.002305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d862bd-9678-4b0d-94fb-2acec9b918ed admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:58.003014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d862bd-9678-4b0d-94fb-2acec9b918ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:58.004952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d862bd-9678-4b0d-94fb-2acec9b918ed admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:58.014891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae44b0-29f2-4a08-998c-abbac5c6717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:58.015798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae44b0-29f2-4a08-998c-abbac5c6717e admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:58.016081 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae44b0-29f2-4a08-998c-abbac5c6717e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:58.016355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae44b0-29f2-4a08-998c-abbac5c6717e admin admin] PATH_INFO: `/v3/projects/1177bbbd1faf430a8b269593ca6ab0c3/users/f222756fc39a4daabea5c299a8e5f41e/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:58.027949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1070/2122] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:57 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:58.032905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae44b0-29f2-4a08-998c-abbac5c6717e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1177bbbd1faf430a8b269593ca6ab0c3, user_id=f222756fc39a4daabea5c299a8e5f41e, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:58.034483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d862bd-9678-4b0d-94fb-2acec9b918ed admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:58.035648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae44b0-29f2-4a08-998c-abbac5c6717e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:58.048467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d862bd-9678-4b0d-94fb-2acec9b918ed admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:58.052525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1053/2123] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:57 2022] PUT /identity/v3/projects/1177bbbd1faf430a8b269593ca6ab0c3/users/f222756fc39a4daabea5c299a8e5f41e/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:58.057881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91603fd-25ee-4708-9661-c16ed2c62e96 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:58.058277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91603fd-25ee-4708-9661-c16ed2c62e96 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:58.058647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91603fd-25ee-4708-9661-c16ed2c62e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:58.075574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a91603fd-25ee-4708-9661-c16ed2c62e96 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:58.076870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a91603fd-25ee-4708-9661-c16ed2c62e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:58.107454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538564e-87e6-4c3d-babd-f0ebb30c7994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:58.108037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538564e-87e6-4c3d-babd-f0ebb30c7994 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:58.108319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538564e-87e6-4c3d-babd-f0ebb30c7994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:58.108593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538564e-87e6-4c3d-babd-f0ebb30c7994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:58.109320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538564e-87e6-4c3d-babd-f0ebb30c7994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:58.110943 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538564e-87e6-4c3d-babd-f0ebb30c7994 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:58.120993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1071/2124] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:58.125566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538564e-87e6-4c3d-babd-f0ebb30c7994 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:58.135668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538564e-87e6-4c3d-babd-f0ebb30c7994 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:58.137839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1054/2125] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:21:58.143482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af295588-6a3c-48cf-bdcb-b3e1d1e11908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:58.144361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af295588-6a3c-48cf-bdcb-b3e1d1e11908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:58.144639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af295588-6a3c-48cf-bdcb-b3e1d1e11908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:58.144904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af295588-6a3c-48cf-bdcb-b3e1d1e11908 admin admin] PATH_INFO: `/v3/projects/4a7c7b30b9bd4821a75beca185c91028/users/5fbc96d1894e4a9caa79278947fb018d/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:58.148192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91603fd-25ee-4708-9661-c16ed2c62e96 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:58.150416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a91603fd-25ee-4708-9661-c16ed2c62e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:58.161346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af295588-6a3c-48cf-bdcb-b3e1d1e11908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a7c7b30b9bd4821a75beca185c91028, user_id=5fbc96d1894e4a9caa79278947fb018d, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:58.164446 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af295588-6a3c-48cf-bdcb-b3e1d1e11908 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:58.177286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1072/2126] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:58 2022] PUT /identity/v3/projects/4a7c7b30b9bd4821a75beca185c91028/users/5fbc96d1894e4a9caa79278947fb018d/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:58.181782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af295588-6a3c-48cf-bdcb-b3e1d1e11908 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:58.199215 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:58.205998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:58.211871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91603fd-25ee-4708-9661-c16ed2c62e96 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:58.251082 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af295588-6a3c-48cf-bdcb-b3e1d1e11908 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:58.256581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679a6ca-9244-4de0-80f4-1254a2bbb264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:58.257513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679a6ca-9244-4de0-80f4-1254a2bbb264 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:58.257828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679a6ca-9244-4de0-80f4-1254a2bbb264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:58.258102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679a6ca-9244-4de0-80f4-1254a2bbb264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:58.258829 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679a6ca-9244-4de0-80f4-1254a2bbb264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:58.260423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679a6ca-9244-4de0-80f4-1254a2bbb264 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:58.266013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bf36e-4938-4489-81f5-b78e2425c18c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:58.266687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bf36e-4938-4489-81f5-b78e2425c18c service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:58.267009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bf36e-4938-4489-81f5-b78e2425c18c service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:58.267288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bf36e-4938-4489-81f5-b78e2425c18c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:58.271192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1073/2127] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:21:58.275462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679a6ca-9244-4de0-80f4-1254a2bbb264 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:58.278227 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bf36e-4938-4489-81f5-b78e2425c18c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:58.279818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bf36e-4938-4489-81f5-b78e2425c18c service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:58.284903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679a6ca-9244-4de0-80f4-1254a2bbb264 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:58.290705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1055/2128] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:58.290915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f454f7-98d7-4fdd-a9cd-fb44f2c4f04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:58.291657 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f454f7-98d7-4fdd-a9cd-fb44f2c4f04f admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:58.291944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f454f7-98d7-4fdd-a9cd-fb44f2c4f04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:58.292220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f454f7-98d7-4fdd-a9cd-fb44f2c4f04f admin admin] PATH_INFO: `/v3/projects/4a7c7b30b9bd4821a75beca185c91028/users/5fbc96d1894e4a9caa79278947fb018d/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:58.294218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f454f7-98d7-4fdd-a9cd-fb44f2c4f04f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a7c7b30b9bd4821a75beca185c91028, user_id=5fbc96d1894e4a9caa79278947fb018d, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:58.296889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f454f7-98d7-4fdd-a9cd-fb44f2c4f04f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:58.309747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1074/2129] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:58 2022] PUT /identity/v3/projects/4a7c7b30b9bd4821a75beca185c91028/users/5fbc96d1894e4a9caa79278947fb018d/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:59.079667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0be3bd-99c4-4221-9420-7310eb807ad8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:59.080191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0be3bd-99c4-4221-9420-7310eb807ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:59.080675 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0be3bd-99c4-4221-9420-7310eb807ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:59.132188 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3e0be3bd-99c4-4221-9420-7310eb807ad8 None None] MFA Rules not processed for user `f222756fc39a4daabea5c299a8e5f41e`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:59.133445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3e0be3bd-99c4-4221-9420-7310eb807ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:59.247135 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1056/2130] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:21:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:59.251342 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f454f7-98d7-4fdd-a9cd-fb44f2c4f04f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:59.328805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f454f7-98d7-4fdd-a9cd-fb44f2c4f04f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:59.335600 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dccb8-ff2f-43b2-a3ab-281946d610ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:59.336622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dccb8-ff2f-43b2-a3ab-281946d610ae admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:59.336622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dccb8-ff2f-43b2-a3ab-281946d610ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:59.337010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dccb8-ff2f-43b2-a3ab-281946d610ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:59.338137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03dccb8-ff2f-43b2-a3ab-281946d610ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:59.340088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03dccb8-ff2f-43b2-a3ab-281946d610ae admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:59.374698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1075/2131] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:21:59 2022] POST /identity/v3/projects => generated 405 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:59.378635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0be3bd-99c4-4221-9420-7310eb807ad8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:59.391735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0be3bd-99c4-4221-9420-7310eb807ad8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:59.464044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7a43f-e1dc-4608-aef1-3a6a1ef79d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:59.464760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7a43f-e1dc-4608-aef1-3a6a1ef79d00 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:59.465048 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7a43f-e1dc-4608-aef1-3a6a1ef79d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:59.465328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7a43f-e1dc-4608-aef1-3a6a1ef79d00 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:59.466057 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7a43f-e1dc-4608-aef1-3a6a1ef79d00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:59.468010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7a43f-e1dc-4608-aef1-3a6a1ef79d00 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:59.492904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1057/2132] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:21:59 2022] POST /identity/v3/users => generated 309 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:21:59.496955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dccb8-ff2f-43b2-a3ab-281946d610ae admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:59.507766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dccb8-ff2f-43b2-a3ab-281946d610ae admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:59.514121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54077f2-90a7-4d34-93c7-a245450491bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:59.514950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54077f2-90a7-4d34-93c7-a245450491bc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:59.515242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54077f2-90a7-4d34-93c7-a245450491bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:59.515603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54077f2-90a7-4d34-93c7-a245450491bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:59.516462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54077f2-90a7-4d34-93c7-a245450491bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:59.518310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54077f2-90a7-4d34-93c7-a245450491bc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:59.540093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1076/2133] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:59.541090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7a43f-e1dc-4608-aef1-3a6a1ef79d00 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:59.552380 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7a43f-e1dc-4608-aef1-3a6a1ef79d00 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:59.558971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14c96a-c24d-43dc-af0a-558682230f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:59.559944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14c96a-c24d-43dc-af0a-558682230f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:59.560349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14c96a-c24d-43dc-af0a-558682230f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:59.560630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14c96a-c24d-43dc-af0a-558682230f83 admin admin] PATH_INFO: `/v3/projects/871c3728c40d4db5b249240d65fe8415/users/25d1cb0c248e4eaf937262db7305e8e8/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:59.578958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14c96a-c24d-43dc-af0a-558682230f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871c3728c40d4db5b249240d65fe8415, user_id=25d1cb0c248e4eaf937262db7305e8e8, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:59.581649 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14c96a-c24d-43dc-af0a-558682230f83 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:59.591038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:59.597924 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1058/2134] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:59 2022] PUT /identity/v3/projects/871c3728c40d4db5b249240d65fe8415/users/25d1cb0c248e4eaf937262db7305e8e8/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:21:59.602373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54077f2-90a7-4d34-93c7-a245450491bc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:59.615991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54077f2-90a7-4d34-93c7-a245450491bc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:59.686335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f80fe-9f34-4db7-b095-df5f152a498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:59.687489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f80fe-9f34-4db7-b095-df5f152a498c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:59.687783 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f80fe-9f34-4db7-b095-df5f152a498c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:59.688066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f80fe-9f34-4db7-b095-df5f152a498c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:59.688943 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f80fe-9f34-4db7-b095-df5f152a498c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:59.690808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f80fe-9f34-4db7-b095-df5f152a498c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:59.702703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1077/2135] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:21:59.706375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14c96a-c24d-43dc-af0a-558682230f83 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:21:59.718562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14c96a-c24d-43dc-af0a-558682230f83 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:21:59.725583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9775ab-1862-45ce-9aea-43b77bd375d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:21:59.726754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9775ab-1862-45ce-9aea-43b77bd375d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:59.727042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9775ab-1862-45ce-9aea-43b77bd375d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:59.727322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9775ab-1862-45ce-9aea-43b77bd375d6 admin admin] PATH_INFO: `/v3/projects/871c3728c40d4db5b249240d65fe8415/users/25d1cb0c248e4eaf937262db7305e8e8/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:59.730202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9775ab-1862-45ce-9aea-43b77bd375d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871c3728c40d4db5b249240d65fe8415, user_id=25d1cb0c248e4eaf937262db7305e8e8, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:21:59.732901 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9775ab-1862-45ce-9aea-43b77bd375d6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:21:59.745331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1059/2136] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:21:59 2022] PUT /identity/v3/projects/871c3728c40d4db5b249240d65fe8415/users/25d1cb0c248e4eaf937262db7305e8e8/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:22:00.088942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f80fe-9f34-4db7-b095-df5f152a498c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:00.091962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-936f80fe-9f34-4db7-b095-df5f152a498c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:00.118845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f80fe-9f34-4db7-b095-df5f152a498c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:00.153834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:00.235897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:00.236835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:00.237120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:00.237472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] PATH_INFO: `/v3/users/baf5566de2654de18e0fa38756fb72e2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:00.239272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf5566de2654de18e0fa38756fb72e2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:00.242110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:00.338659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf5566de2654de18e0fa38756fb72e2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:00.340153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf5566de2654de18e0fa38756fb72e2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:00.340877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf5566de2654de18e0fa38756fb72e2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:00.363761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf5566de2654de18e0fa38756fb72e2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:00.375938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf5566de2654de18e0fa38756fb72e2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:00.402384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1078/2137] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:22:00 2022] DELETE /identity/v3/users/baf5566de2654de18e0fa38756fb72e2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:00.406907 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6cc1b-84b9-435c-99d7-318f239af91a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:00.407408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6cc1b-84b9-435c-99d7-318f239af91a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:00.407668 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6cc1b-84b9-435c-99d7-318f239af91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:00.439696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-22d6cc1b-84b9-435c-99d7-318f239af91a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:00.441078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-22d6cc1b-84b9-435c-99d7-318f239af91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:00.590905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1060/2138] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:00.608057 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:00.618051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:00.750251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583aebc-6bda-4273-91dc-a997181bb587 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:00.754404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b647d7-0b58-4ab4-980e-119485d4e200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:00.755649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b647d7-0b58-4ab4-980e-119485d4e200 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:00.756450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b647d7-0b58-4ab4-980e-119485d4e200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:00.756743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b647d7-0b58-4ab4-980e-119485d4e200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:00.797957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b647d7-0b58-4ab4-980e-119485d4e200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:00.801820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b647d7-0b58-4ab4-980e-119485d4e200 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:00.823666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d1504-38ef-4cfd-af3d-322302167044 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:00.824582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d1504-38ef-4cfd-af3d-322302167044 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:00.825121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d1504-38ef-4cfd-af3d-322302167044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:00.826438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1079/2139] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:22:00.862094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0d1d1504-38ef-4cfd-af3d-322302167044 None None] MFA Rules not processed for user `25d1cb0c248e4eaf937262db7305e8e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:00.863799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0d1d1504-38ef-4cfd-af3d-322302167044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:00.978336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1061/2140] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:00 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:01.581518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b647d7-0b58-4ab4-980e-119485d4e200 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:01.585097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b8b647d7-0b58-4ab4-980e-119485d4e200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:01.619312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b647d7-0b58-4ab4-980e-119485d4e200 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:01.630056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:01.633341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:01.634058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:01.634553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] PATH_INFO: `/v3/projects/24ea377e9c51407ca742c5ce93e6c274` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:01.641079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24ea377e9c51407ca742c5ce93e6c274)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:01.649033 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:01.764741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:01.797341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24ea377e9c51407ca742c5ce93e6c274', 'request_id': 'req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:01.816352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24ea377e9c51407ca742c5ce93e6c274', 'request_id': 'req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:01.838587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24ea377e9c51407ca742c5ce93e6c274', 'request_id': 'req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:01.862656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] The token cache is being invalidate because project 24ea377e9c51407ca742c5ce93e6c274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:22:01.863135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:01.866164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1dc5fbf5-1819-45d6-a6c4-91c58e33f2a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:01.871417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1080/2141] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:22:01 2022] DELETE /identity/v3/projects/24ea377e9c51407ca742c5ce93e6c274 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:04.230320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d1504-38ef-4cfd-af3d-322302167044 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:04.231855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0d1d1504-38ef-4cfd-af3d-322302167044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:04.275202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d1504-38ef-4cfd-af3d-322302167044 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:04.366293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:04.367182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:04.367471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:04.367747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] PATH_INFO: `/v3/users/f222756fc39a4daabea5c299a8e5f41e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:04.369155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f222756fc39a4daabea5c299a8e5f41e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:04.371163 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:04.390162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:04.458546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f222756fc39a4daabea5c299a8e5f41e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:04.472583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f222756fc39a4daabea5c299a8e5f41e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:04.473450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f222756fc39a4daabea5c299a8e5f41e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:04.490243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f222756fc39a4daabea5c299a8e5f41e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:04.514674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f222756fc39a4daabea5c299a8e5f41e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:04.537993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1062/2142] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:22:04 2022] DELETE /identity/v3/users/f222756fc39a4daabea5c299a8e5f41e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:04.767330 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f2ad2-eb6b-4d36-9508-348f6a429bed None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:04.768311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f2ad2-eb6b-4d36-9508-348f6a429bed None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:04.768812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f2ad2-eb6b-4d36-9508-348f6a429bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:04.803549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ad2f2ad2-eb6b-4d36-9508-348f6a429bed None None] MFA Rules not processed for user `5fbc96d1894e4a9caa79278947fb018d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:04.804889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ad2f2ad2-eb6b-4d36-9508-348f6a429bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:04.902152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1081/2143] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:04.914329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:04.920022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:04.939599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f2ad2-eb6b-4d36-9508-348f6a429bed None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:04.941453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ad2f2ad2-eb6b-4d36-9508-348f6a429bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:05.024044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f5b89-b0e7-436b-b799-6868e7a2c932 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:05.029916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731e5b7-452b-441a-a025-c95e23a910c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:05.030574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731e5b7-452b-441a-a025-c95e23a910c2 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:05.030860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731e5b7-452b-441a-a025-c95e23a910c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:05.031141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731e5b7-452b-441a-a025-c95e23a910c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:05.032732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f2ad2-eb6b-4d36-9508-348f6a429bed None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:05.032919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3731e5b7-452b-441a-a025-c95e23a910c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:05.038138 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:05.039218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:05.039502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:05.039772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] PATH_INFO: `/v3/projects/1177bbbd1faf430a8b269593ca6ab0c3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:05.041286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1177bbbd1faf430a8b269593ca6ab0c3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:05.043381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:05.100951 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:05.102907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1177bbbd1faf430a8b269593ca6ab0c3', 'request_id': 'req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.112350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731e5b7-452b-441a-a025-c95e23a910c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:05.113408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1177bbbd1faf430a8b269593ca6ab0c3', 'request_id': 'req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.114280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731e5b7-452b-441a-a025-c95e23a910c2 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:05.123395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1177bbbd1faf430a8b269593ca6ab0c3', 'request_id': 'req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.129817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1063/2144] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:22:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:22:05.137438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] The token cache is being invalidate because project 1177bbbd1faf430a8b269593ca6ab0c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:22:05.137763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.138436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.140005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1082/2145] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:22:04 2022] DELETE /identity/v3/projects/1177bbbd1faf430a8b269593ca6ab0c3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:22:05.143908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731e5b7-452b-441a-a025-c95e23a910c2 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:05.156628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731e5b7-452b-441a-a025-c95e23a910c2 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:05.198936 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:05.246579 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf15ad-9683-4951-a6c0-3b6173cf31ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:05.247737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf15ad-9683-4951-a6c0-3b6173cf31ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:05.248021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf15ad-9683-4951-a6c0-3b6173cf31ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:05.248275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf15ad-9683-4951-a6c0-3b6173cf31ab admin admin] PATH_INFO: `/v3/users/25d1cb0c248e4eaf937262db7305e8e8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:05.249689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf15ad-9683-4951-a6c0-3b6173cf31ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d1cb0c248e4eaf937262db7305e8e8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:05.252205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf15ad-9683-4951-a6c0-3b6173cf31ab admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:05.314704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9cbf15ad-9683-4951-a6c0-3b6173cf31ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d1cb0c248e4eaf937262db7305e8e8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.323287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9cbf15ad-9683-4951-a6c0-3b6173cf31ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d1cb0c248e4eaf937262db7305e8e8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.324059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9cbf15ad-9683-4951-a6c0-3b6173cf31ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d1cb0c248e4eaf937262db7305e8e8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.339511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9cbf15ad-9683-4951-a6c0-3b6173cf31ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d1cb0c248e4eaf937262db7305e8e8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.347770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9cbf15ad-9683-4951-a6c0-3b6173cf31ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d1cb0c248e4eaf937262db7305e8e8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.357535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1064/2146] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:22:05 2022] DELETE /identity/v3/users/25d1cb0c248e4eaf937262db7305e8e8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:05.580213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:05.581853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:05.599040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c07f3-f5dc-412d-a48f-2767e3f8f61a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:05.691142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:05.692162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:05.692572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:05.692994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] PATH_INFO: `/v3/projects/871c3728c40d4db5b249240d65fe8415` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:05.694710 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] RBAC: Authorizing `identity:delete_project(project_id=871c3728c40d4db5b249240d65fe8415)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:05.696807 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:05.752931 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871c3728c40d4db5b249240d65fe8415', 'request_id': 'req-8b28a905-1cb6-451b-b435-346fd58bf76e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.759690 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871c3728c40d4db5b249240d65fe8415', 'request_id': 'req-8b28a905-1cb6-451b-b435-346fd58bf76e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.768308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871c3728c40d4db5b249240d65fe8415', 'request_id': 'req-8b28a905-1cb6-451b-b435-346fd58bf76e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.780123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] The token cache is being invalidate because project 871c3728c40d4db5b249240d65fe8415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:22:05.780414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.781085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8b28a905-1cb6-451b-b435-346fd58bf76e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:05.782972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1083/2147] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:22:05 2022] DELETE /identity/v3/projects/871c3728c40d4db5b249240d65fe8415 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:05.787999 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3df6b-b7c5-46f7-af81-43f68f6d440b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:05.788315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3df6b-b7c5-46f7-af81-43f68f6d440b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:05.788594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3df6b-b7c5-46f7-af81-43f68f6d440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:05.815222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cae3df6b-b7c5-46f7-af81-43f68f6d440b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:05.816616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cae3df6b-b7c5-46f7-af81-43f68f6d440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:05.889524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1065/2148] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:05.897669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75272018-6fcb-4bd5-806f-32fed420dbcf None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:05.898006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75272018-6fcb-4bd5-806f-32fed420dbcf None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:05.898289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75272018-6fcb-4bd5-806f-32fed420dbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:05.917161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-75272018-6fcb-4bd5-806f-32fed420dbcf None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:05.918423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-75272018-6fcb-4bd5-806f-32fed420dbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:05.926796 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1084/2149] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:05.931740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3df6b-b7c5-46f7-af81-43f68f6d440b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:05.946270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3df6b-b7c5-46f7-af81-43f68f6d440b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:05.951347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422b763-3da9-4e82-88a1-bb4819461a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:05.952042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422b763-3da9-4e82-88a1-bb4819461a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:05.952403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422b763-3da9-4e82-88a1-bb4819461a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:05.952745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422b763-3da9-4e82-88a1-bb4819461a84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:05.953561 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422b763-3da9-4e82-88a1-bb4819461a84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:22:05.954152 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422b763-3da9-4e82-88a1-bb4819461a84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:05.955862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422b763-3da9-4e82-88a1-bb4819461a84 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:05.969807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1066/2150] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:22:05.975780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75272018-6fcb-4bd5-806f-32fed420dbcf None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:05.990850 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75272018-6fcb-4bd5-806f-32fed420dbcf None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:06.001240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3183ba-66e6-4fc0-a909-7d85fb84b6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:06.002035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3183ba-66e6-4fc0-a909-7d85fb84b6ba admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.002327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3183ba-66e6-4fc0-a909-7d85fb84b6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.002608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3183ba-66e6-4fc0-a909-7d85fb84b6ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.003344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3183ba-66e6-4fc0-a909-7d85fb84b6ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:06.005613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3183ba-66e6-4fc0-a909-7d85fb84b6ba admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:06.037757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1085/2151] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:22:05 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:22:06.043726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422b763-3da9-4e82-88a1-bb4819461a84 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:06.056145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422b763-3da9-4e82-88a1-bb4819461a84 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:06.121229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:06.176943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473581c9-1e89-4649-adaa-c8c013a4ce4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:06.178331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473581c9-1e89-4649-adaa-c8c013a4ce4c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.178630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473581c9-1e89-4649-adaa-c8c013a4ce4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.178910 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473581c9-1e89-4649-adaa-c8c013a4ce4c admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.179612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473581c9-1e89-4649-adaa-c8c013a4ce4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:06.181848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473581c9-1e89-4649-adaa-c8c013a4ce4c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:06.206175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1067/2152] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:22:06 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:22:06.206990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3183ba-66e6-4fc0-a909-7d85fb84b6ba admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:06.219567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3183ba-66e6-4fc0-a909-7d85fb84b6ba admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:06.226342 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d7ebb-f3c4-4e27-9798-3197b108975f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:06.227060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7ebb-f3c4-4e27-9798-3197b108975f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.227345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7ebb-f3c4-4e27-9798-3197b108975f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.227619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7ebb-f3c4-4e27-9798-3197b108975f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.228528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d7ebb-f3c4-4e27-9798-3197b108975f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:06.230005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d7ebb-f3c4-4e27-9798-3197b108975f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:06.249417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1086/2153] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:22:06.251965 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473581c9-1e89-4649-adaa-c8c013a4ce4c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:06.262409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473581c9-1e89-4649-adaa-c8c013a4ce4c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:06.267829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de8f47-6e51-44a3-95c9-35075cf2027f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:06.268515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de8f47-6e51-44a3-95c9-35075cf2027f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.268803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de8f47-6e51-44a3-95c9-35075cf2027f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.269079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de8f47-6e51-44a3-95c9-35075cf2027f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.269947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de8f47-6e51-44a3-95c9-35075cf2027f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:06.272696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de8f47-6e51-44a3-95c9-35075cf2027f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:06.285741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1068/2154] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:22:06.290305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d7ebb-f3c4-4e27-9798-3197b108975f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:06.301636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d7ebb-f3c4-4e27-9798-3197b108975f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:06.307978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d0e14-fa6c-459c-8ef5-ae1f4b557551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:06.308929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d0e14-fa6c-459c-8ef5-ae1f4b557551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.309242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d0e14-fa6c-459c-8ef5-ae1f4b557551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.309477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d0e14-fa6c-459c-8ef5-ae1f4b557551 admin admin] PATH_INFO: `/v3/projects/d20c7d9f8ecb4b98b73209e1f574506c/users/322c28dfbde64a338e26af6b6ffc54f8/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.331979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d0e14-fa6c-459c-8ef5-ae1f4b557551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d20c7d9f8ecb4b98b73209e1f574506c, user_id=322c28dfbde64a338e26af6b6ffc54f8, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:06.336113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d0e14-fa6c-459c-8ef5-ae1f4b557551 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:06.351981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1087/2155] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:22:06 2022] PUT /identity/v3/projects/d20c7d9f8ecb4b98b73209e1f574506c/users/322c28dfbde64a338e26af6b6ffc54f8/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:22:06.362258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126495b0-a53c-404a-862b-3cd07e896a38 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.362475 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126495b0-a53c-404a-862b-3cd07e896a38 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.362626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126495b0-a53c-404a-862b-3cd07e896a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.392923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-126495b0-a53c-404a-862b-3cd07e896a38 None None] MFA Rules not processed for user `322c28dfbde64a338e26af6b6ffc54f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:06.394275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-126495b0-a53c-404a-862b-3cd07e896a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:06.473770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1069/2156] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:06.475891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d0e14-fa6c-459c-8ef5-ae1f4b557551 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:06.480509 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:06.559495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d0e14-fa6c-459c-8ef5-ae1f4b557551 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:06.564613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1e4c4-f2f6-498f-a0dd-c765441df2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:06.565410 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1e4c4-f2f6-498f-a0dd-c765441df2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.565876 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1e4c4-f2f6-498f-a0dd-c765441df2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.566235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1e4c4-f2f6-498f-a0dd-c765441df2b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.567187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1e4c4-f2f6-498f-a0dd-c765441df2b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:06.569355 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1e4c4-f2f6-498f-a0dd-c765441df2b9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:06.599906 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1088/2157] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:22:06 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:22:06.604155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126495b0-a53c-404a-862b-3cd07e896a38 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:06.616564 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126495b0-a53c-404a-862b-3cd07e896a38 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:06.684174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b1d6b-b49c-4dc7-8956-0f71329eafce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:06.684896 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b1d6b-b49c-4dc7-8956-0f71329eafce admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.685278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b1d6b-b49c-4dc7-8956-0f71329eafce admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.685647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b1d6b-b49c-4dc7-8956-0f71329eafce admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.686452 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b1d6b-b49c-4dc7-8956-0f71329eafce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:06.688503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b1d6b-b49c-4dc7-8956-0f71329eafce admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:06.718606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1070/2158] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:22:06 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:22:06.722900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1e4c4-f2f6-498f-a0dd-c765441df2b9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:06.735279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1e4c4-f2f6-498f-a0dd-c765441df2b9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:06.741374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1092ecd-0807-4f9a-a25e-7644bfa54a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:06.742155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1092ecd-0807-4f9a-a25e-7644bfa54a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.742432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1092ecd-0807-4f9a-a25e-7644bfa54a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.742707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1092ecd-0807-4f9a-a25e-7644bfa54a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.743444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1092ecd-0807-4f9a-a25e-7644bfa54a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:06.745054 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1092ecd-0807-4f9a-a25e-7644bfa54a4b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:06.758567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1089/2159] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:22:06.763319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b1d6b-b49c-4dc7-8956-0f71329eafce admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:06.776057 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b1d6b-b49c-4dc7-8956-0f71329eafce admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:06.781628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452988bc-7acc-45ec-9ba7-35080e3eb4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:06.782548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452988bc-7acc-45ec-9ba7-35080e3eb4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.782970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452988bc-7acc-45ec-9ba7-35080e3eb4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.783254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452988bc-7acc-45ec-9ba7-35080e3eb4b8 admin admin] PATH_INFO: `/v3/projects/4b347a967db3495e967ee9d8081fa64f/users/ef6392077b814666b256bdd0a9af7b96/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.799836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452988bc-7acc-45ec-9ba7-35080e3eb4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b347a967db3495e967ee9d8081fa64f, user_id=ef6392077b814666b256bdd0a9af7b96, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:06.803299 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452988bc-7acc-45ec-9ba7-35080e3eb4b8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:06.831998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1071/2160] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:22:06 2022] PUT /identity/v3/projects/4b347a967db3495e967ee9d8081fa64f/users/ef6392077b814666b256bdd0a9af7b96/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:06.836115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1092ecd-0807-4f9a-a25e-7644bfa54a4b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:06.850615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1092ecd-0807-4f9a-a25e-7644bfa54a4b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:06.917191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e091f2f-3ae3-48cd-b260-355d4496b6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:06.918206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e091f2f-3ae3-48cd-b260-355d4496b6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.918675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e091f2f-3ae3-48cd-b260-355d4496b6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.918954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e091f2f-3ae3-48cd-b260-355d4496b6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.920014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e091f2f-3ae3-48cd-b260-355d4496b6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:06.921309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e091f2f-3ae3-48cd-b260-355d4496b6d0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:06.933185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1090/2161] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:22:06.938918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452988bc-7acc-45ec-9ba7-35080e3eb4b8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:06.950959 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452988bc-7acc-45ec-9ba7-35080e3eb4b8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:06.955984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bd5ec-c8b1-4d75-bb7d-1a5ea6fbc684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:06.956889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bd5ec-c8b1-4d75-bb7d-1a5ea6fbc684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.957170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bd5ec-c8b1-4d75-bb7d-1a5ea6fbc684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.957448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bd5ec-c8b1-4d75-bb7d-1a5ea6fbc684 admin admin] PATH_INFO: `/v3/projects/4b347a967db3495e967ee9d8081fa64f/users/ef6392077b814666b256bdd0a9af7b96/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:06.960490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227bd5ec-c8b1-4d75-bb7d-1a5ea6fbc684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b347a967db3495e967ee9d8081fa64f, user_id=ef6392077b814666b256bdd0a9af7b96, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:06.963505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227bd5ec-c8b1-4d75-bb7d-1a5ea6fbc684 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:06.966533 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:06.988497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1072/2162] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:22:06 2022] PUT /identity/v3/projects/4b347a967db3495e967ee9d8081fa64f/users/ef6392077b814666b256bdd0a9af7b96/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:06.990905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45557d5c-3078-44b4-8a2b-22c67e8c7932 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:06.991196 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45557d5c-3078-44b4-8a2b-22c67e8c7932 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:06.991479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45557d5c-3078-44b4-8a2b-22c67e8c7932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:07.028572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-45557d5c-3078-44b4-8a2b-22c67e8c7932 None None] MFA Rules not processed for user `ef6392077b814666b256bdd0a9af7b96`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:07.029834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-45557d5c-3078-44b4-8a2b-22c67e8c7932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:07.127699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:07.158295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1091/2163] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:07.166544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bd5ec-c8b1-4d75-bb7d-1a5ea6fbc684 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:07.168575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-227bd5ec-c8b1-4d75-bb7d-1a5ea6fbc684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:07.267318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bd5ec-c8b1-4d75-bb7d-1a5ea6fbc684 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:07.274310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce7e38-1c48-4471-9b23-fdd8ca36fb5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:07.274983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce7e38-1c48-4471-9b23-fdd8ca36fb5b service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:07.275266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce7e38-1c48-4471-9b23-fdd8ca36fb5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:07.275537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce7e38-1c48-4471-9b23-fdd8ca36fb5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:07.287893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dce7e38-1c48-4471-9b23-fdd8ca36fb5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:07.290184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dce7e38-1c48-4471-9b23-fdd8ca36fb5b service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:07.301584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1073/2164] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:22:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:22:07.420190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45557d5c-3078-44b4-8a2b-22c67e8c7932 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:07.431901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45557d5c-3078-44b4-8a2b-22c67e8c7932 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:07.444385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1aa2fa-7dd9-4074-a086-95dc566c6668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:07.444967 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aa2fa-7dd9-4074-a086-95dc566c6668 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:07.445248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aa2fa-7dd9-4074-a086-95dc566c6668 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:07.445520 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aa2fa-7dd9-4074-a086-95dc566c6668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:07.515958 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1aa2fa-7dd9-4074-a086-95dc566c6668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:07.517541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1aa2fa-7dd9-4074-a086-95dc566c6668 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:07.529500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1092/2165] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:22:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:22:14.256161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce7e38-1c48-4471-9b23-fdd8ca36fb5b service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:14.257590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3dce7e38-1c48-4471-9b23-fdd8ca36fb5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:14.403002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce7e38-1c48-4471-9b23-fdd8ca36fb5b service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:14.408612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:14.409501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:14.409822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:14.410102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] PATH_INFO: `/v3/users/9ca7b8174734486784b728086b139e94` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:14.411320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca7b8174734486784b728086b139e94)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:14.414486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:14.466015 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=1,keystone:SELECT=94,keystone:DELETE=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:14.478906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca7b8174734486784b728086b139e94'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:14.487867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca7b8174734486784b728086b139e94'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:14.488713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca7b8174734486784b728086b139e94'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:14.503164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca7b8174734486784b728086b139e94'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:14.511514 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca7b8174734486784b728086b139e94'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:14.518155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1074/2166] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:22:14 2022] DELETE /identity/v3/users/9ca7b8174734486784b728086b139e94 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:14.524228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a88054-718d-4008-bc92-e0f11a213a70 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:14.524720 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a88054-718d-4008-bc92-e0f11a213a70 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:14.524812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a88054-718d-4008-bc92-e0f11a213a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:14.560381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-00a88054-718d-4008-bc92-e0f11a213a70 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:14.561590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-00a88054-718d-4008-bc92-e0f11a213a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:14.627578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1093/2167] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:14.640401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:14.642928 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:14.751351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56334eb1-a73e-4820-bd1e-08cb8047ead1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:14.755085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913ef9f-249c-43c2-ac84-1d8fe8a306cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:14.756343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913ef9f-249c-43c2-ac84-1d8fe8a306cc service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:14.756616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913ef9f-249c-43c2-ac84-1d8fe8a306cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:14.756887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913ef9f-249c-43c2-ac84-1d8fe8a306cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:14.766329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913ef9f-249c-43c2-ac84-1d8fe8a306cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:14.767765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913ef9f-249c-43c2-ac84-1d8fe8a306cc service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:14.778932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1075/2168] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:22:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:22:15.077291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a88054-718d-4008-bc92-e0f11a213a70 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:15.078885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-00a88054-718d-4008-bc92-e0f11a213a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:15.120374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a88054-718d-4008-bc92-e0f11a213a70 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:15.126974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:15.127970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:15.128546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:15.128900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] PATH_INFO: `/v3/projects/f09afeed1caa455781da41ea97b2426f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:15.132485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f09afeed1caa455781da41ea97b2426f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:15.135103 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:15.197040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f09afeed1caa455781da41ea97b2426f', 'request_id': 'req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:15.204044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f09afeed1caa455781da41ea97b2426f', 'request_id': 'req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:15.212037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f09afeed1caa455781da41ea97b2426f', 'request_id': 'req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:15.222697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] The token cache is being invalidate because project f09afeed1caa455781da41ea97b2426f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:22:15.223385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:15.223776 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-346433bf-e4a0-4adc-8e3c-17fa6f0c3c27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:15.225491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1094/2169] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:22:15 2022] DELETE /identity/v3/projects/f09afeed1caa455781da41ea97b2426f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:15.230706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788cc02-9dca-43e8-aa3c-26d965203c74 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:15.231023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788cc02-9dca-43e8-aa3c-26d965203c74 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:15.231295 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788cc02-9dca-43e8-aa3c-26d965203c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:15.272710 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f788cc02-9dca-43e8-aa3c-26d965203c74 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:15.273949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f788cc02-9dca-43e8-aa3c-26d965203c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:15.337067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1076/2170] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:15.344882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02dd7d-77dc-44dc-89cc-b18fd1db7403 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:15.345115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02dd7d-77dc-44dc-89cc-b18fd1db7403 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:15.345440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02dd7d-77dc-44dc-89cc-b18fd1db7403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:15.364457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6d02dd7d-77dc-44dc-89cc-b18fd1db7403 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:15.366199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6d02dd7d-77dc-44dc-89cc-b18fd1db7403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:15.374188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1095/2171] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:15.378525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788cc02-9dca-43e8-aa3c-26d965203c74 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:15.390153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788cc02-9dca-43e8-aa3c-26d965203c74 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:15.394858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7a3a2-cc34-4b39-a11c-c1ef953f8a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:15.395519 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7a3a2-cc34-4b39-a11c-c1ef953f8a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:15.395800 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7a3a2-cc34-4b39-a11c-c1ef953f8a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:15.396327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7a3a2-cc34-4b39-a11c-c1ef953f8a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:15.397264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7a3a2-cc34-4b39-a11c-c1ef953f8a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:22:15.397574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7a3a2-cc34-4b39-a11c-c1ef953f8a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:15.399254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7a3a2-cc34-4b39-a11c-c1ef953f8a1a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:15.400962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:15.414657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1077/2172] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:22:15.418950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02dd7d-77dc-44dc-89cc-b18fd1db7403 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:15.428754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02dd7d-77dc-44dc-89cc-b18fd1db7403 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:15.433455 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194077a7-d43f-4971-bd8b-ea057551699a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:15.434137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194077a7-d43f-4971-bd8b-ea057551699a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:15.434423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194077a7-d43f-4971-bd8b-ea057551699a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:15.434690 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194077a7-d43f-4971-bd8b-ea057551699a admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:15.435378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194077a7-d43f-4971-bd8b-ea057551699a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:15.437289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194077a7-d43f-4971-bd8b-ea057551699a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:15.440696 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:15.469649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1096/2173] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:22:15 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:22:15.470743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7a3a2-cc34-4b39-a11c-c1ef953f8a1a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:15.481307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7a3a2-cc34-4b39-a11c-c1ef953f8a1a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:15.540169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fa84e-4b11-455e-bf8b-839d8679894b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:15.540848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fa84e-4b11-455e-bf8b-839d8679894b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:15.541137 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fa84e-4b11-455e-bf8b-839d8679894b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:15.541421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fa84e-4b11-455e-bf8b-839d8679894b admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:15.542155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fa84e-4b11-455e-bf8b-839d8679894b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:15.544094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fa84e-4b11-455e-bf8b-839d8679894b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:15.565098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1078/2174] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:22:15 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:22:15.569429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194077a7-d43f-4971-bd8b-ea057551699a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:15.579977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194077a7-d43f-4971-bd8b-ea057551699a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:15.584689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da3a07-6c59-4213-be3b-061dc41b4d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:15.585357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da3a07-6c59-4213-be3b-061dc41b4d0b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:15.585644 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da3a07-6c59-4213-be3b-061dc41b4d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:15.585948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da3a07-6c59-4213-be3b-061dc41b4d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:15.586716 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da3a07-6c59-4213-be3b-061dc41b4d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:15.588316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da3a07-6c59-4213-be3b-061dc41b4d0b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:15.598989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1097/2175] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:22:15.606741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fa84e-4b11-455e-bf8b-839d8679894b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:15.617552 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fa84e-4b11-455e-bf8b-839d8679894b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:15.622340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d0908-8bef-4e1a-945b-35254694b364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:15.623363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d0908-8bef-4e1a-945b-35254694b364 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:15.623363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d0908-8bef-4e1a-945b-35254694b364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:15.623589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d0908-8bef-4e1a-945b-35254694b364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:15.624287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d0908-8bef-4e1a-945b-35254694b364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:15.625932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d0908-8bef-4e1a-945b-35254694b364 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:15.639744 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1079/2176] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:22:15.640871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da3a07-6c59-4213-be3b-061dc41b4d0b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:15.651744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da3a07-6c59-4213-be3b-061dc41b4d0b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:15.658518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb0c08-ba62-408e-86b4-a45f06f1050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:15.659453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb0c08-ba62-408e-86b4-a45f06f1050a admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:15.659740 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb0c08-ba62-408e-86b4-a45f06f1050a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:15.660017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb0c08-ba62-408e-86b4-a45f06f1050a admin admin] PATH_INFO: `/v3/projects/a4cb58e89ebe471ea2ce7471a4036982/users/86d409153bdb4738903eecb131a9df60/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:15.676414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb0c08-ba62-408e-86b4-a45f06f1050a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4cb58e89ebe471ea2ce7471a4036982, user_id=86d409153bdb4738903eecb131a9df60, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:15.679776 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb0c08-ba62-408e-86b4-a45f06f1050a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:15.693225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1098/2177] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:22:15 2022] PUT /identity/v3/projects/a4cb58e89ebe471ea2ce7471a4036982/users/86d409153bdb4738903eecb131a9df60/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:15.704349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1507f3-0230-4ef4-b637-e87073410893 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:15.704645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1507f3-0230-4ef4-b637-e87073410893 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:15.704929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1507f3-0230-4ef4-b637-e87073410893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:15.731699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1c1507f3-0230-4ef4-b637-e87073410893 None None] MFA Rules not processed for user `86d409153bdb4738903eecb131a9df60`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:15.732936 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1c1507f3-0230-4ef4-b637-e87073410893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:15.797292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1080/2178] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:15.809027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb0c08-ba62-408e-86b4-a45f06f1050a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:15.810793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-baeb0c08-ba62-408e-86b4-a45f06f1050a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:15.868174 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb0c08-ba62-408e-86b4-a45f06f1050a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:15.871818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a372e-d264-4f42-ad60-6a78ccf3f1f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:15.872461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a372e-d264-4f42-ad60-6a78ccf3f1f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:15.872742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a372e-d264-4f42-ad60-6a78ccf3f1f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:15.873016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a372e-d264-4f42-ad60-6a78ccf3f1f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:15.881588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a372e-d264-4f42-ad60-6a78ccf3f1f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:15.883053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a372e-d264-4f42-ad60-6a78ccf3f1f9 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:15.891579 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1099/2179] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:22:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:22:25.794434 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:25.887641 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:25.966728 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1507f3-0230-4ef4-b637-e87073410893 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:26.031979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1507f3-0230-4ef4-b637-e87073410893 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:26.036463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:26.037736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:26.038034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:26.038308 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] PATH_INFO: `/v3/users/86d409153bdb4738903eecb131a9df60` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:26.039486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=86d409153bdb4738903eecb131a9df60)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:26.041391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:26.088840 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86d409153bdb4738903eecb131a9df60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:26.097244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86d409153bdb4738903eecb131a9df60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:26.097923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86d409153bdb4738903eecb131a9df60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:26.110709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86d409153bdb4738903eecb131a9df60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:26.118121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86d409153bdb4738903eecb131a9df60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:26.124693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1081/2180] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:22:25 2022] DELETE /identity/v3/users/86d409153bdb4738903eecb131a9df60 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:26.130207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31ed68-5045-45c0-af0a-b701c11f974a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:26.130496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31ed68-5045-45c0-af0a-b701c11f974a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:26.130866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31ed68-5045-45c0-af0a-b701c11f974a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:26.148753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1e31ed68-5045-45c0-af0a-b701c11f974a None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:26.150079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1e31ed68-5045-45c0-af0a-b701c11f974a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:26.214710 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1100/2181] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:26.225395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:26.226819 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:26.286595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7a799-4f45-41c5-b1a7-2af221a436ca admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:26.290749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813d60b-da8a-4d62-a350-dbacd6066921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:26.291409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813d60b-da8a-4d62-a350-dbacd6066921 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:26.291696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813d60b-da8a-4d62-a350-dbacd6066921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:26.291974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813d60b-da8a-4d62-a350-dbacd6066921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:26.301174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813d60b-da8a-4d62-a350-dbacd6066921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:26.303291 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813d60b-da8a-4d62-a350-dbacd6066921 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:26.312676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1082/2182] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:22:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:22:26.581542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31ed68-5045-45c0-af0a-b701c11f974a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:26.583513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1e31ed68-5045-45c0-af0a-b701c11f974a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:26.595308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31ed68-5045-45c0-af0a-b701c11f974a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:26.601060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:26.602878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:26.603516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:26.603801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] PATH_INFO: `/v3/projects/a4cb58e89ebe471ea2ce7471a4036982` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:26.605121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4cb58e89ebe471ea2ce7471a4036982)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:26.607691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:26.659725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4cb58e89ebe471ea2ce7471a4036982', 'request_id': 'req-dc1c3fbf-8011-4cef-8783-c7a6d85631df'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:26.666881 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4cb58e89ebe471ea2ce7471a4036982', 'request_id': 'req-dc1c3fbf-8011-4cef-8783-c7a6d85631df'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:26.674040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4cb58e89ebe471ea2ce7471a4036982', 'request_id': 'req-dc1c3fbf-8011-4cef-8783-c7a6d85631df'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:26.684150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] The token cache is being invalidate because project a4cb58e89ebe471ea2ce7471a4036982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:22:26.684437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:26.685060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dc1c3fbf-8011-4cef-8783-c7a6d85631df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:26.686641 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1101/2183] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:22:26 2022] DELETE /identity/v3/projects/a4cb58e89ebe471ea2ce7471a4036982 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:36.308489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:36.681179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:38.578357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c0a68-b419-4d67-996f-ddd3278c296e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:38.578876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c0a68-b419-4d67-996f-ddd3278c296e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:38.578986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c0a68-b419-4d67-996f-ddd3278c296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:38.596530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f87c0a68-b419-4d67-996f-ddd3278c296e None None] MFA Rules not processed for user `c935f035c757482ca912a8c4b77714a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:38.597771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f87c0a68-b419-4d67-996f-ddd3278c296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:38.663533 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1083/2184] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:40.966175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d3872-b3c3-43ba-824f-364f66e77d28 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:40.967478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d3872-b3c3-43ba-824f-364f66e77d28 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:40.967478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d3872-b3c3-43ba-824f-364f66e77d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:40.990888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e44d3872-b3c3-43ba-824f-364f66e77d28 None None] MFA Rules not processed for user `b781c4c6cb61439fb781988a7da028ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:40.992925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e44d3872-b3c3-43ba-824f-364f66e77d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:41.053511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1102/2185] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:22:48.656375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=25 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:51.045621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c0a68-b419-4d67-996f-ddd3278c296e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:51.046698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=25 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:51.047209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f87c0a68-b419-4d67-996f-ddd3278c296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:51.115773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c0a68-b419-4d67-996f-ddd3278c296e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:51.120960 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:51.121874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:51.122173 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:51.122433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] PATH_INFO: `/v3/users/322c28dfbde64a338e26af6b6ffc54f8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:51.123611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=322c28dfbde64a338e26af6b6ffc54f8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:51.125529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:51.181042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '322c28dfbde64a338e26af6b6ffc54f8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.187870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '322c28dfbde64a338e26af6b6ffc54f8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.188487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '322c28dfbde64a338e26af6b6ffc54f8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.200728 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '322c28dfbde64a338e26af6b6ffc54f8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.207465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '322c28dfbde64a338e26af6b6ffc54f8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.213458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1084/2186] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:22:51 2022] DELETE /identity/v3/users/322c28dfbde64a338e26af6b6ffc54f8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:51.217960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cba51-8352-4b15-9f33-76eac6ee1fd9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:51.218256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cba51-8352-4b15-9f33-76eac6ee1fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:51.218537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cba51-8352-4b15-9f33-76eac6ee1fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:51.235742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1a8cba51-8352-4b15-9f33-76eac6ee1fd9 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:51.236941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1a8cba51-8352-4b15-9f33-76eac6ee1fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:51.310730 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1103/2187] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:51.320110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:51.321557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:51.380205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec2fc7-3625-43a1-b56f-61748f7345c8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:51.384107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2751b67-46e1-4d5d-887f-6fb21fa1182f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:51.384748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2751b67-46e1-4d5d-887f-6fb21fa1182f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:51.385034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2751b67-46e1-4d5d-887f-6fb21fa1182f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:51.385325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2751b67-46e1-4d5d-887f-6fb21fa1182f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:51.394598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2751b67-46e1-4d5d-887f-6fb21fa1182f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:51.396032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2751b67-46e1-4d5d-887f-6fb21fa1182f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:51.406057 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1085/2188] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:22:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:22:51.690327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cba51-8352-4b15-9f33-76eac6ee1fd9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:51.692100 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1a8cba51-8352-4b15-9f33-76eac6ee1fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:51.706615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cba51-8352-4b15-9f33-76eac6ee1fd9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:51.713756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:51.715462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:51.715892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:51.716321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] PATH_INFO: `/v3/projects/d20c7d9f8ecb4b98b73209e1f574506c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:51.717841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d20c7d9f8ecb4b98b73209e1f574506c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:51.720025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:51.777702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd20c7d9f8ecb4b98b73209e1f574506c', 'request_id': 'req-66f84458-e55c-43bc-9035-87d202ebc9aa'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.786345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd20c7d9f8ecb4b98b73209e1f574506c', 'request_id': 'req-66f84458-e55c-43bc-9035-87d202ebc9aa'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.797058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd20c7d9f8ecb4b98b73209e1f574506c', 'request_id': 'req-66f84458-e55c-43bc-9035-87d202ebc9aa'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.809863 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] The token cache is being invalidate because project d20c7d9f8ecb4b98b73209e1f574506c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:22:51.810157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.810815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.812380 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1104/2189] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:22:51 2022] DELETE /identity/v3/projects/d20c7d9f8ecb4b98b73209e1f574506c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:51.816465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2751b67-46e1-4d5d-887f-6fb21fa1182f service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:51.827046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2751b67-46e1-4d5d-887f-6fb21fa1182f service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:51.850491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:51.898212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a021c-2b61-47e5-8e17-4e436d2b30e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:51.899270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a021c-2b61-47e5-8e17-4e436d2b30e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:51.899559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a021c-2b61-47e5-8e17-4e436d2b30e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:51.899839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a021c-2b61-47e5-8e17-4e436d2b30e0 admin admin] PATH_INFO: `/v3/users/ef6392077b814666b256bdd0a9af7b96` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:51.901157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a021c-2b61-47e5-8e17-4e436d2b30e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6392077b814666b256bdd0a9af7b96)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:51.903127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a021c-2b61-47e5-8e17-4e436d2b30e0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:51.954212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b74a021c-2b61-47e5-8e17-4e436d2b30e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6392077b814666b256bdd0a9af7b96'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.964175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b74a021c-2b61-47e5-8e17-4e436d2b30e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6392077b814666b256bdd0a9af7b96'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.965182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b74a021c-2b61-47e5-8e17-4e436d2b30e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6392077b814666b256bdd0a9af7b96'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.984888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b74a021c-2b61-47e5-8e17-4e436d2b30e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6392077b814666b256bdd0a9af7b96'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:51.993678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b74a021c-2b61-47e5-8e17-4e436d2b30e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6392077b814666b256bdd0a9af7b96'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:52.004188 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1086/2190] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:22:51 2022] DELETE /identity/v3/users/ef6392077b814666b256bdd0a9af7b96 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:52.403732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:52.405672 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:52.423402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f84458-e55c-43bc-9035-87d202ebc9aa admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:52.532518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:52.533494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:52.533895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:52.534250 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] PATH_INFO: `/v3/projects/4b347a967db3495e967ee9d8081fa64f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:52.535599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b347a967db3495e967ee9d8081fa64f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:52.537835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:52.573106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:52.596036 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b347a967db3495e967ee9d8081fa64f', 'request_id': 'req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:52.603505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b347a967db3495e967ee9d8081fa64f', 'request_id': 'req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:52.612409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b347a967db3495e967ee9d8081fa64f', 'request_id': 'req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:52.625518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] The token cache is being invalidate because project 4b347a967db3495e967ee9d8081fa64f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:22:52.626196 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:52.626730 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fb116116-2fb7-4dbc-825e-33ce4c5b97a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:52.628539 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1105/2191] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:22:52 2022] DELETE /identity/v3/projects/4b347a967db3495e967ee9d8081fa64f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:52.634771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37e34e-fddc-4289-ae5e-1057707a4e3e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:52.635390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37e34e-fddc-4289-ae5e-1057707a4e3e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:52.635390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37e34e-fddc-4289-ae5e-1057707a4e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:52.669040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6c37e34e-fddc-4289-ae5e-1057707a4e3e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:52.670303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6c37e34e-fddc-4289-ae5e-1057707a4e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:52.744520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1087/2192] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:52.752464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312d5b0-1b7f-4490-a238-469693cccde2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:52.752787 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312d5b0-1b7f-4490-a238-469693cccde2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:52.753812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312d5b0-1b7f-4490-a238-469693cccde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:52.774900 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7312d5b0-1b7f-4490-a238-469693cccde2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:52.776749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7312d5b0-1b7f-4490-a238-469693cccde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:52.795227 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1106/2193] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:52.798389 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37e34e-fddc-4289-ae5e-1057707a4e3e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:52.809765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37e34e-fddc-4289-ae5e-1057707a4e3e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:52.814565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c543933-357e-4481-81c5-3003190cc639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:52.815234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543933-357e-4481-81c5-3003190cc639 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:52.815513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543933-357e-4481-81c5-3003190cc639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:52.815785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543933-357e-4481-81c5-3003190cc639 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:52.816542 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c543933-357e-4481-81c5-3003190cc639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:22:52.816845 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c543933-357e-4481-81c5-3003190cc639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:52.818490 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c543933-357e-4481-81c5-3003190cc639 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:52.832999 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1088/2194] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:22:52.837513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312d5b0-1b7f-4490-a238-469693cccde2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:52.847894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312d5b0-1b7f-4490-a238-469693cccde2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:52.853159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90f69c-b1e0-4c9b-9ac5-767e058fe4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:52.853999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90f69c-b1e0-4c9b-9ac5-767e058fe4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:52.854289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90f69c-b1e0-4c9b-9ac5-767e058fe4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:52.854563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90f69c-b1e0-4c9b-9ac5-767e058fe4b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:52.855270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90f69c-b1e0-4c9b-9ac5-767e058fe4b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:52.857396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90f69c-b1e0-4c9b-9ac5-767e058fe4b4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:52.893545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1107/2195] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:22:52 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:22:52.897946 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c543933-357e-4481-81c5-3003190cc639 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:52.908795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c543933-357e-4481-81c5-3003190cc639 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:52.937623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:53.012655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72525b-5d21-4d0d-9a5d-144615949074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:53.013621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72525b-5d21-4d0d-9a5d-144615949074 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.014392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72525b-5d21-4d0d-9a5d-144615949074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.014880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72525b-5d21-4d0d-9a5d-144615949074 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.016171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72525b-5d21-4d0d-9a5d-144615949074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:53.020023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72525b-5d21-4d0d-9a5d-144615949074 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:53.046476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1089/2196] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:22:52 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:22:53.053020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90f69c-b1e0-4c9b-9ac5-767e058fe4b4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:53.067404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90f69c-b1e0-4c9b-9ac5-767e058fe4b4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:53.073766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494049cb-d02d-47a6-8c00-42bc87daae8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:53.074545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494049cb-d02d-47a6-8c00-42bc87daae8e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.074878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494049cb-d02d-47a6-8c00-42bc87daae8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.075217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494049cb-d02d-47a6-8c00-42bc87daae8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.076018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494049cb-d02d-47a6-8c00-42bc87daae8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:53.077789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494049cb-d02d-47a6-8c00-42bc87daae8e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:53.092333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1108/2197] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:22:53.098622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72525b-5d21-4d0d-9a5d-144615949074 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:53.109441 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72525b-5d21-4d0d-9a5d-144615949074 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:53.114132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cec048-5f05-4251-ad90-c742429c5f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:53.114817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cec048-5f05-4251-ad90-c742429c5f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.115102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cec048-5f05-4251-ad90-c742429c5f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.115374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cec048-5f05-4251-ad90-c742429c5f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.116106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cec048-5f05-4251-ad90-c742429c5f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:53.117893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cec048-5f05-4251-ad90-c742429c5f8f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:53.131776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1090/2198] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:22:53.136618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494049cb-d02d-47a6-8c00-42bc87daae8e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:53.154384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494049cb-d02d-47a6-8c00-42bc87daae8e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:53.160453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9328ae6-a640-4828-a4a1-4625b6c9dde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:53.161453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9328ae6-a640-4828-a4a1-4625b6c9dde1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.161846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9328ae6-a640-4828-a4a1-4625b6c9dde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.162220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9328ae6-a640-4828-a4a1-4625b6c9dde1 admin admin] PATH_INFO: `/v3/projects/db1f645c078346c0a1160e623c6d81cb/users/3899caa80b014a50acd107b19f849e49/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.191934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9328ae6-a640-4828-a4a1-4625b6c9dde1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1f645c078346c0a1160e623c6d81cb, user_id=3899caa80b014a50acd107b19f849e49, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:53.195800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9328ae6-a640-4828-a4a1-4625b6c9dde1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:53.212573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1109/2199] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:22:53 2022] PUT /identity/v3/projects/db1f645c078346c0a1160e623c6d81cb/users/3899caa80b014a50acd107b19f849e49/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:53.223812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66140c8e-d79a-467a-a70f-efa3741f8e2f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.224149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66140c8e-d79a-467a-a70f-efa3741f8e2f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.224370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66140c8e-d79a-467a-a70f-efa3741f8e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.253446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-66140c8e-d79a-467a-a70f-efa3741f8e2f None None] MFA Rules not processed for user `3899caa80b014a50acd107b19f849e49`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:53.254934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-66140c8e-d79a-467a-a70f-efa3741f8e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:53.333753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1091/2200] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:53.339181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9328ae6-a640-4828-a4a1-4625b6c9dde1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:53.416513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9328ae6-a640-4828-a4a1-4625b6c9dde1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:53.423121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43524f10-26cc-4336-b062-4f00cf4ce7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:53.423904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524f10-26cc-4336-b062-4f00cf4ce7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.425995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524f10-26cc-4336-b062-4f00cf4ce7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.425995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524f10-26cc-4336-b062-4f00cf4ce7a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.426217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43524f10-26cc-4336-b062-4f00cf4ce7a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:53.428136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43524f10-26cc-4336-b062-4f00cf4ce7a6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:53.461837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1110/2201] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:22:53 2022] POST /identity/v3/projects => generated 393 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:22:53.466311 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66140c8e-d79a-467a-a70f-efa3741f8e2f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:53.477666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66140c8e-d79a-467a-a70f-efa3741f8e2f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:53.550556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab60475-5f91-4a93-b828-22e3af2ad1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:53.551512 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab60475-5f91-4a93-b828-22e3af2ad1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.551869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab60475-5f91-4a93-b828-22e3af2ad1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.552219 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab60475-5f91-4a93-b828-22e3af2ad1a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.553486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab60475-5f91-4a93-b828-22e3af2ad1a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:53.557046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab60475-5f91-4a93-b828-22e3af2ad1a3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:53.590414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43524f10-26cc-4336-b062-4f00cf4ce7a6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:53.591161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1092/2202] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:22:53 2022] POST /identity/v3/users => generated 303 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:22:53.601618 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43524f10-26cc-4336-b062-4f00cf4ce7a6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:53.607570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223056c9-176a-4117-a122-3b9e55247c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:53.608503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223056c9-176a-4117-a122-3b9e55247c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.608805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223056c9-176a-4117-a122-3b9e55247c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.609071 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223056c9-176a-4117-a122-3b9e55247c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.609865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223056c9-176a-4117-a122-3b9e55247c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:53.611465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223056c9-176a-4117-a122-3b9e55247c5e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:53.631331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab60475-5f91-4a93-b828-22e3af2ad1a3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:53.642243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab60475-5f91-4a93-b828-22e3af2ad1a3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:53.643151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1111/2203] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:22:53.655639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50218db6-8f1c-4364-913e-fbd19511cf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:53.658273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50218db6-8f1c-4364-913e-fbd19511cf1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.658912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50218db6-8f1c-4364-913e-fbd19511cf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.659158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50218db6-8f1c-4364-913e-fbd19511cf1c admin admin] PATH_INFO: `/v3/projects/416ce392a775471cbdd9cba0314f7df3/users/a9f932da5d9d4bafb3b8e5be72096bae/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.688053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50218db6-8f1c-4364-913e-fbd19511cf1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=416ce392a775471cbdd9cba0314f7df3, user_id=a9f932da5d9d4bafb3b8e5be72096bae, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:53.691191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50218db6-8f1c-4364-913e-fbd19511cf1c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:53.693482 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:53.722393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1093/2204] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:22:53 2022] PUT /identity/v3/projects/416ce392a775471cbdd9cba0314f7df3/users/a9f932da5d9d4bafb3b8e5be72096bae/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:53.726288 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223056c9-176a-4117-a122-3b9e55247c5e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:53.737959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223056c9-176a-4117-a122-3b9e55247c5e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:53.761460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:53.774275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5896dd5-b8ca-4d1f-ac70-f3bbbeba9fea None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.774574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5896dd5-b8ca-4d1f-ac70-f3bbbeba9fea None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.774859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5896dd5-b8ca-4d1f-ac70-f3bbbeba9fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.795647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b5896dd5-b8ca-4d1f-ac70-f3bbbeba9fea None None] MFA Rules not processed for user `30de794b71b343289491531dfbe37f53`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:53.796886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b5896dd5-b8ca-4d1f-ac70-f3bbbeba9fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:53.814498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff965e-d103-4735-a0de-e00d637e990f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:53.815318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff965e-d103-4735-a0de-e00d637e990f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.815608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff965e-d103-4735-a0de-e00d637e990f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.815920 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff965e-d103-4735-a0de-e00d637e990f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.817221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff965e-d103-4735-a0de-e00d637e990f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:53.820093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff965e-d103-4735-a0de-e00d637e990f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:53.834463 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1112/2205] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:22:53.836540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff965e-d103-4735-a0de-e00d637e990f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:53.852467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff965e-d103-4735-a0de-e00d637e990f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:53.858207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae56fff-4baf-42b3-bc5b-e8732a064fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:53.859757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae56fff-4baf-42b3-bc5b-e8732a064fae admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.860390 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae56fff-4baf-42b3-bc5b-e8732a064fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.860797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae56fff-4baf-42b3-bc5b-e8732a064fae admin admin] PATH_INFO: `/v3/projects/416ce392a775471cbdd9cba0314f7df3/users/a9f932da5d9d4bafb3b8e5be72096bae/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.861860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1094/2206] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:53.863136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae56fff-4baf-42b3-bc5b-e8732a064fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=416ce392a775471cbdd9cba0314f7df3, user_id=a9f932da5d9d4bafb3b8e5be72096bae, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:53.866985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae56fff-4baf-42b3-bc5b-e8732a064fae admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:53.880986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1113/2207] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:22:53 2022] PUT /identity/v3/projects/416ce392a775471cbdd9cba0314f7df3/users/a9f932da5d9d4bafb3b8e5be72096bae/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:22:53.891695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e01ec-657c-419b-8171-ad2ba8be85b1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:53.891988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e01ec-657c-419b-8171-ad2ba8be85b1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:53.892268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e01ec-657c-419b-8171-ad2ba8be85b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:53.924055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-153e01ec-657c-419b-8171-ad2ba8be85b1 None None] MFA Rules not processed for user `a9f932da5d9d4bafb3b8e5be72096bae`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:53.925292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-153e01ec-657c-419b-8171-ad2ba8be85b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:53.999183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1095/2208] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:54.010409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae56fff-4baf-42b3-bc5b-e8732a064fae admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:54.012128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8ae56fff-4baf-42b3-bc5b-e8732a064fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:54.095592 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae56fff-4baf-42b3-bc5b-e8732a064fae admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:54.099401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b3083-517f-4c65-8401-3d618da3d122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:54.100056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3083-517f-4c65-8401-3d618da3d122 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:54.100341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3083-517f-4c65-8401-3d618da3d122 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:54.100619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3083-517f-4c65-8401-3d618da3d122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:54.113508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b3083-517f-4c65-8401-3d618da3d122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:54.115259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b3083-517f-4c65-8401-3d618da3d122 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:54.127798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1114/2209] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:22:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:22:54.178590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e01ec-657c-419b-8171-ad2ba8be85b1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:54.180839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-153e01ec-657c-419b-8171-ad2ba8be85b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:54.193323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e01ec-657c-419b-8171-ad2ba8be85b1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:54.197748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261e58b-e830-4b2a-8c34-db6249ca5c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:54.198415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e58b-e830-4b2a-8c34-db6249ca5c04 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:54.198701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e58b-e830-4b2a-8c34-db6249ca5c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:54.198982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e58b-e830-4b2a-8c34-db6249ca5c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:54.263953 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261e58b-e830-4b2a-8c34-db6249ca5c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:54.265413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261e58b-e830-4b2a-8c34-db6249ca5c04 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:54.275776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1096/2210] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:22:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:22:57.684926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b3083-517f-4c65-8401-3d618da3d122 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:57.776120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b3083-517f-4c65-8401-3d618da3d122 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:57.780836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:57.781759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:57.782040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:57.782312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] PATH_INFO: `/v3/users/3899caa80b014a50acd107b19f849e49` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:57.783526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3899caa80b014a50acd107b19f849e49)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:57.785443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:57.826735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:57.844619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3899caa80b014a50acd107b19f849e49'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:57.845289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3899caa80b014a50acd107b19f849e49'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:57.846001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3899caa80b014a50acd107b19f849e49'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:57.858262 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3899caa80b014a50acd107b19f849e49'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:57.865158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3899caa80b014a50acd107b19f849e49'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:57.878834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1115/2211] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:22:57 2022] DELETE /identity/v3/users/3899caa80b014a50acd107b19f849e49 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:57.884198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076c43a-20b1-401b-9e2e-334fd0c9762e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:57.884649 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076c43a-20b1-401b-9e2e-334fd0c9762e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:57.884763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076c43a-20b1-401b-9e2e-334fd0c9762e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:57.901974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f076c43a-20b1-401b-9e2e-334fd0c9762e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:22:57.903165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f076c43a-20b1-401b-9e2e-334fd0c9762e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:57.920535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:57.971564 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1097/2212] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:22:57.993673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:57.995663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:58.062689 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660c36e-520d-4b19-84a9-c1d3fed8d97f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:58.067161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7ab7f-9928-4b7a-ae77-a325c96143a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:58.067810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7ab7f-9928-4b7a-ae77-a325c96143a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:58.068683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7ab7f-9928-4b7a-ae77-a325c96143a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:58.068683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7ab7f-9928-4b7a-ae77-a325c96143a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:58.078343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7ab7f-9928-4b7a-ae77-a325c96143a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:58.079782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7ab7f-9928-4b7a-ae77-a325c96143a4 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:58.091161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1116/2213] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:22:58.490048 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076c43a-20b1-401b-9e2e-334fd0c9762e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:58.492727 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f076c43a-20b1-401b-9e2e-334fd0c9762e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:58.523053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076c43a-20b1-401b-9e2e-334fd0c9762e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:58.557421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:58.558303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:58.558613 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:58.558862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] PATH_INFO: `/v3/projects/db1f645c078346c0a1160e623c6d81cb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:58.560985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db1f645c078346c0a1160e623c6d81cb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:58.563041 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:58.645609 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db1f645c078346c0a1160e623c6d81cb', 'request_id': 'req-6b9588b0-c6ab-49c7-96f1-1c656e101290'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:58.653369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db1f645c078346c0a1160e623c6d81cb', 'request_id': 'req-6b9588b0-c6ab-49c7-96f1-1c656e101290'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:58.660554 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db1f645c078346c0a1160e623c6d81cb', 'request_id': 'req-6b9588b0-c6ab-49c7-96f1-1c656e101290'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:58.678225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] The token cache is being invalidate because project db1f645c078346c0a1160e623c6d81cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:22:58.678537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:58.680147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:58.682136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1098/2214] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:22:58 2022] DELETE /identity/v3/projects/db1f645c078346c0a1160e623c6d81cb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:58.686046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7ab7f-9928-4b7a-ae77-a325c96143a4 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:58.696183 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7ab7f-9928-4b7a-ae77-a325c96143a4 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:58.780588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118303f0-d97b-4797-9b9a-f007bd57fe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:58.781645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118303f0-d97b-4797-9b9a-f007bd57fe51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:58.782084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118303f0-d97b-4797-9b9a-f007bd57fe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:58.782436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118303f0-d97b-4797-9b9a-f007bd57fe51 admin admin] PATH_INFO: `/v3/users/a9f932da5d9d4bafb3b8e5be72096bae` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:58.784063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118303f0-d97b-4797-9b9a-f007bd57fe51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9f932da5d9d4bafb3b8e5be72096bae)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:58.786392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118303f0-d97b-4797-9b9a-f007bd57fe51 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:58.880862 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-118303f0-d97b-4797-9b9a-f007bd57fe51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9f932da5d9d4bafb3b8e5be72096bae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:58.882561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-118303f0-d97b-4797-9b9a-f007bd57fe51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9f932da5d9d4bafb3b8e5be72096bae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:58.883791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-118303f0-d97b-4797-9b9a-f007bd57fe51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9f932da5d9d4bafb3b8e5be72096bae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:58.893761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:58.905946 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-118303f0-d97b-4797-9b9a-f007bd57fe51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9f932da5d9d4bafb3b8e5be72096bae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:58.918772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-118303f0-d97b-4797-9b9a-f007bd57fe51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9f932da5d9d4bafb3b8e5be72096bae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:58.940184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1117/2215] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:22:58 2022] DELETE /identity/v3/users/a9f932da5d9d4bafb3b8e5be72096bae => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:22:59.753664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:22:59.755150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:22:59.779142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9588b0-c6ab-49c7-96f1-1c656e101290 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:22:59.892999 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:22:59.896866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:22:59.896866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:22:59.896866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] PATH_INFO: `/v3/projects/416ce392a775471cbdd9cba0314f7df3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:22:59.898933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] RBAC: Authorizing `identity:delete_project(project_id=416ce392a775471cbdd9cba0314f7df3)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:22:59.906967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:22:59.987261 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:22:59.988799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '416ce392a775471cbdd9cba0314f7df3', 'request_id': 'req-50c84cd6-4bc3-462e-9dc2-438e3e25045d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:22:59.998287 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '416ce392a775471cbdd9cba0314f7df3', 'request_id': 'req-50c84cd6-4bc3-462e-9dc2-438e3e25045d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:00.010305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '416ce392a775471cbdd9cba0314f7df3', 'request_id': 'req-50c84cd6-4bc3-462e-9dc2-438e3e25045d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:00.026745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] The token cache is being invalidate because project 416ce392a775471cbdd9cba0314f7df3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:23:00.027116 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:00.028495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-50c84cd6-4bc3-462e-9dc2-438e3e25045d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:00.030536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1099/2216] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:22:59 2022] DELETE /identity/v3/projects/416ce392a775471cbdd9cba0314f7df3 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:23:00.036811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1232c30-5a1d-4a22-825e-3c9ded9864f4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:00.037568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1232c30-5a1d-4a22-825e-3c9ded9864f4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:00.037656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1232c30-5a1d-4a22-825e-3c9ded9864f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:00.078421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f1232c30-5a1d-4a22-825e-3c9ded9864f4 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:00.079785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f1232c30-5a1d-4a22-825e-3c9ded9864f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:00.200884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1118/2217] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:23:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:23:00.215289 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47525-bcc1-4fb5-853a-d0bbe4097c40 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:00.215583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47525-bcc1-4fb5-853a-d0bbe4097c40 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:00.215864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47525-bcc1-4fb5-853a-d0bbe4097c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:00.246396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9ca47525-bcc1-4fb5-853a-d0bbe4097c40 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:00.247834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ca47525-bcc1-4fb5-853a-d0bbe4097c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:00.268761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1100/2218] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:23:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:23:00.270054 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1232c30-5a1d-4a22-825e-3c9ded9864f4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:00.283638 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1232c30-5a1d-4a22-825e-3c9ded9864f4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:00.289000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c5f21-d82d-46d9-9f72-fc03f42a1fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:00.289674 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c5f21-d82d-46d9-9f72-fc03f42a1fee admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:00.290008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c5f21-d82d-46d9-9f72-fc03f42a1fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:00.290286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c5f21-d82d-46d9-9f72-fc03f42a1fee admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:00.291054 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c5f21-d82d-46d9-9f72-fc03f42a1fee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:23:00.291364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c5f21-d82d-46d9-9f72-fc03f42a1fee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:00.293105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c5f21-d82d-46d9-9f72-fc03f42a1fee admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:00.306854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1119/2219] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:23:00.310988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca47525-bcc1-4fb5-853a-d0bbe4097c40 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:00.321255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca47525-bcc1-4fb5-853a-d0bbe4097c40 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:00.326334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96fc2a-032f-4b26-bae4-ea67de4b1011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:00.327000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96fc2a-032f-4b26-bae4-ea67de4b1011 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:00.327285 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96fc2a-032f-4b26-bae4-ea67de4b1011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:00.327565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96fc2a-032f-4b26-bae4-ea67de4b1011 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:00.328266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96fc2a-032f-4b26-bae4-ea67de4b1011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:00.330223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96fc2a-032f-4b26-bae4-ea67de4b1011 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:00.362073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1101/2220] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:23:00 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:23:00.363237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c5f21-d82d-46d9-9f72-fc03f42a1fee admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:00.374059 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c5f21-d82d-46d9-9f72-fc03f42a1fee admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:00.433679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cff18-9b83-41a6-8ed7-ccdd41ab3aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:00.434558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cff18-9b83-41a6-8ed7-ccdd41ab3aca admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:00.434894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cff18-9b83-41a6-8ed7-ccdd41ab3aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:00.435244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cff18-9b83-41a6-8ed7-ccdd41ab3aca admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:00.436012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cff18-9b83-41a6-8ed7-ccdd41ab3aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:00.438152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cff18-9b83-41a6-8ed7-ccdd41ab3aca admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:00.467468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1120/2221] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:23:00 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:23:00.468678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96fc2a-032f-4b26-bae4-ea67de4b1011 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:00.480092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96fc2a-032f-4b26-bae4-ea67de4b1011 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:00.484797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8bdc2-26de-4cbe-9595-5429c606f878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:00.485503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8bdc2-26de-4cbe-9595-5429c606f878 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:00.485782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8bdc2-26de-4cbe-9595-5429c606f878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:00.486067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8bdc2-26de-4cbe-9595-5429c606f878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:00.486799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8bdc2-26de-4cbe-9595-5429c606f878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:00.488418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8bdc2-26de-4cbe-9595-5429c606f878 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:00.499641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1102/2222] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:23:00.503656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cff18-9b83-41a6-8ed7-ccdd41ab3aca admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:00.514971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cff18-9b83-41a6-8ed7-ccdd41ab3aca admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:00.521507 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11c782-e5a7-46cd-aa1c-86f8682d18c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:00.522940 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11c782-e5a7-46cd-aa1c-86f8682d18c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:00.523228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11c782-e5a7-46cd-aa1c-86f8682d18c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:00.524902 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11c782-e5a7-46cd-aa1c-86f8682d18c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:00.524902 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11c782-e5a7-46cd-aa1c-86f8682d18c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:00.527312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11c782-e5a7-46cd-aa1c-86f8682d18c3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:00.542760 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1121/2223] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:23:00.546702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8bdc2-26de-4cbe-9595-5429c606f878 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:00.560486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8bdc2-26de-4cbe-9595-5429c606f878 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:00.565570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fd990-cc2b-453e-a5ed-4c5202684d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:00.566567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fd990-cc2b-453e-a5ed-4c5202684d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:00.566933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fd990-cc2b-453e-a5ed-4c5202684d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:00.567304 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fd990-cc2b-453e-a5ed-4c5202684d9b admin admin] PATH_INFO: `/v3/projects/4641a4d02b7a40c18baa78bdf652332e/users/90d510673c3d4f038edcbe92aea66487/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:00.585610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fd990-cc2b-453e-a5ed-4c5202684d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4641a4d02b7a40c18baa78bdf652332e, user_id=90d510673c3d4f038edcbe92aea66487, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:00.588596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fd990-cc2b-453e-a5ed-4c5202684d9b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:00.603054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1103/2224] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:23:00 2022] PUT /identity/v3/projects/4641a4d02b7a40c18baa78bdf652332e/users/90d510673c3d4f038edcbe92aea66487/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:23:00.622562 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bf2e6-a965-4a33-b780-6b0b774034a3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:00.623093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bf2e6-a965-4a33-b780-6b0b774034a3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:00.623606 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bf2e6-a965-4a33-b780-6b0b774034a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:00.673022 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4e7bf2e6-a965-4a33-b780-6b0b774034a3 None None] MFA Rules not processed for user `90d510673c3d4f038edcbe92aea66487`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:00.675105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4e7bf2e6-a965-4a33-b780-6b0b774034a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:00.722525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:00.793855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1122/2225] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:23:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:23:00.801347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fd990-cc2b-453e-a5ed-4c5202684d9b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:00.803064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-de8fd990-cc2b-453e-a5ed-4c5202684d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:00.890797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fd990-cc2b-453e-a5ed-4c5202684d9b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:00.894755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45ad4b-2614-4e97-b016-64367c3b4dd3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8386ca93fe48460ba5b6f0c10dc51122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:00.895438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45ad4b-2614-4e97-b016-64367c3b4dd3 service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:00.895720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45ad4b-2614-4e97-b016-64367c3b4dd3 service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:00.895993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45ad4b-2614-4e97-b016-64367c3b4dd3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:00.906057 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45ad4b-2614-4e97-b016-64367c3b4dd3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:00.907528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45ad4b-2614-4e97-b016-64367c3b4dd3 service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:00.918635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1104/2226] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:23:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:04.520598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bf2e6-a965-4a33-b780-6b0b774034a3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:04.522087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4e7bf2e6-a965-4a33-b780-6b0b774034a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:04.589635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bf2e6-a965-4a33-b780-6b0b774034a3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:04.595153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff4292-31a4-4852-ba97-a3eeb523af62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:04.595805 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4292-31a4-4852-ba97-a3eeb523af62 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:04.596088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4292-31a4-4852-ba97-a3eeb523af62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:04.596362 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4292-31a4-4852-ba97-a3eeb523af62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:04.598172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d7ff4292-31a4-4852-ba97-a3eeb523af62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:04.676246 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff4292-31a4-4852-ba97-a3eeb523af62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:04.677702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff4292-31a4-4852-ba97-a3eeb523af62 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:04.688128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1123/2227] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:23:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:08.895740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e3bac-ecfd-42ba-b1e3-9d2ee0bed649 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:08.896254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e3bac-ecfd-42ba-b1e3-9d2ee0bed649 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:08.896583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e3bac-ecfd-42ba-b1e3-9d2ee0bed649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:08.983400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1e8e3bac-ecfd-42ba-b1e3-9d2ee0bed649 None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:08.984659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1e8e3bac-ecfd-42ba-b1e3-9d2ee0bed649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:09.005151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:09.099566 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1105/2228] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:23:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:23:09.175008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff4292-31a4-4852-ba97-a3eeb523af62 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:09.176747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d7ff4292-31a4-4852-ba97-a3eeb523af62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:09.289681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:09.301579 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff4292-31a4-4852-ba97-a3eeb523af62 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:09.312391 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846129c1-47c2-4583-9c7b-cdf8a205bf63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:09.312660 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846129c1-47c2-4583-9c7b-cdf8a205bf63 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:09.312845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846129c1-47c2-4583-9c7b-cdf8a205bf63 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:09.313072 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846129c1-47c2-4583-9c7b-cdf8a205bf63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:09.322964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846129c1-47c2-4583-9c7b-cdf8a205bf63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:09.324758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846129c1-47c2-4583-9c7b-cdf8a205bf63 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:09.354522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1124/2229] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:23:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:23:19.093451 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=22 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:19.330742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:23.436923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e3bac-ecfd-42ba-b1e3-9d2ee0bed649 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:23.438534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1e8e3bac-ecfd-42ba-b1e3-9d2ee0bed649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:23.465330 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846129c1-47c2-4583-9c7b-cdf8a205bf63 service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:23.467794 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-846129c1-47c2-4583-9c7b-cdf8a205bf63 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:23.532683 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e3bac-ecfd-42ba-b1e3-9d2ee0bed649 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:23.538926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e841014-1841-43ce-93e6-673133c0932c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:23.539772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e841014-1841-43ce-93e6-673133c0932c service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:23.540050 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e841014-1841-43ce-93e6-673133c0932c service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:23.540155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846129c1-47c2-4583-9c7b-cdf8a205bf63 service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:23.540343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e841014-1841-43ce-93e6-673133c0932c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:23.542288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0e841014-1841-43ce-93e6-673133c0932c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:23.546937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b02f5-d1e8-43c4-a993-9b41c266f220 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:23.547586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b02f5-d1e8-43c4-a993-9b41c266f220 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:23.547869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b02f5-d1e8-43c4-a993-9b41c266f220 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:23.548141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b02f5-d1e8-43c4-a993-9b41c266f220 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:23.554673 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e841014-1841-43ce-93e6-673133c0932c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:23.556616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e841014-1841-43ce-93e6-673133c0932c service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:23.557271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b02f5-d1e8-43c4-a993-9b41c266f220 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:23.558877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b02f5-d1e8-43c4-a993-9b41c266f220 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:23.569520 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1125/2230] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:23:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:23:23.570435 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1106/2231] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:23:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:23:24.712202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65fafb-f62e-403b-ae9b-6fb0f3767dd7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:24.712817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65fafb-f62e-403b-ae9b-6fb0f3767dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:24.712817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65fafb-f62e-403b-ae9b-6fb0f3767dd7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:24.714166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1126/2232] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:23:24.720314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32be176-ad25-477b-8555-a17382446544 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:24.720883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32be176-ad25-477b-8555-a17382446544 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:24.720992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32be176-ad25-477b-8555-a17382446544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:24.775623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f32be176-ad25-477b-8555-a17382446544 None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:24.776854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f32be176-ad25-477b-8555-a17382446544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:24.861895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1107/2233] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:23:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:23:24.873942 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65fafb-f62e-403b-ae9b-6fb0f3767dd7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:24.875637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8a65fafb-f62e-403b-ae9b-6fb0f3767dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:24.888710 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65fafb-f62e-403b-ae9b-6fb0f3767dd7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:24.892254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2f171-dbcb-4e8b-9981-6119ad4d75a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8386ca93fe48460ba5b6f0c10dc51122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:24.892912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2f171-dbcb-4e8b-9981-6119ad4d75a5 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:24.893195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2f171-dbcb-4e8b-9981-6119ad4d75a5 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:24.893474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2f171-dbcb-4e8b-9981-6119ad4d75a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:24.905479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2f171-dbcb-4e8b-9981-6119ad4d75a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:24.906963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2f171-dbcb-4e8b-9981-6119ad4d75a5 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:24.918962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1127/2234] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:23:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:24.987108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de87f72-23bf-44bf-805f-e69ac96ca70c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:24.987454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de87f72-23bf-44bf-805f-e69ac96ca70c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:24.987742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de87f72-23bf-44bf-805f-e69ac96ca70c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:24.989138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1108/2235] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:23:24.994186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce58220-9b10-4913-9753-66768c86886d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:24.994461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce58220-9b10-4913-9753-66768c86886d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:24.994741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce58220-9b10-4913-9753-66768c86886d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:25.014370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bce58220-9b10-4913-9753-66768c86886d None None] MFA Rules not processed for user `e42c796dbd7441289a7cbc328abeaf81`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:25.015478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bce58220-9b10-4913-9753-66768c86886d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:25.023436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1128/2236] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:23:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:23:25.038183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de87f72-23bf-44bf-805f-e69ac96ca70c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:25.040720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4de87f72-23bf-44bf-805f-e69ac96ca70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:25.058961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de87f72-23bf-44bf-805f-e69ac96ca70c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:25.063369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429549bd-fd1c-4199-8424-3f76496496e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:25.064015 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429549bd-fd1c-4199-8424-3f76496496e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:25.064297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429549bd-fd1c-4199-8424-3f76496496e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:25.064579 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429549bd-fd1c-4199-8424-3f76496496e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:25.072277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429549bd-fd1c-4199-8424-3f76496496e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:25.073701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429549bd-fd1c-4199-8424-3f76496496e4 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:25.082902 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1109/2237] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:23:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:25.170529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a26125-e6ec-4a64-95fe-792fc5d87002 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:25.170820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a26125-e6ec-4a64-95fe-792fc5d87002 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:25.171093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a26125-e6ec-4a64-95fe-792fc5d87002 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:25.172498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1129/2238] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:23:25.178160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9ae8b-151b-4e7c-b179-55c7a9028a4a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:25.178526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9ae8b-151b-4e7c-b179-55c7a9028a4a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:25.179541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9ae8b-151b-4e7c-b179-55c7a9028a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:25.180456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a26125-e6ec-4a64-95fe-792fc5d87002 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:25.200167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e0d9ae8b-151b-4e7c-b179-55c7a9028a4a None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:25.201383 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e0d9ae8b-151b-4e7c-b179-55c7a9028a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:25.211278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1110/2239] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:23:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:23:25.221987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9ae8b-151b-4e7c-b179-55c7a9028a4a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:25.232545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9ae8b-151b-4e7c-b179-55c7a9028a4a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:25.236281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc418cf4-7a18-4208-aac3-ef0848486f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:25.236925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc418cf4-7a18-4208-aac3-ef0848486f06 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:25.237211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc418cf4-7a18-4208-aac3-ef0848486f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:25.237510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc418cf4-7a18-4208-aac3-ef0848486f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:25.247471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:25.250455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc418cf4-7a18-4208-aac3-ef0848486f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:25.250593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a26125-e6ec-4a64-95fe-792fc5d87002 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:25.252093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc418cf4-7a18-4208-aac3-ef0848486f06 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:25.255178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:25.256051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:25.256359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:25.256634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] PATH_INFO: `/v3/users/90d510673c3d4f038edcbe92aea66487` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:25.257908 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90d510673c3d4f038edcbe92aea66487)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:25.259974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:25.263840 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1111/2240] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:23:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:23:25.310939 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90d510673c3d4f038edcbe92aea66487'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:25.311615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90d510673c3d4f038edcbe92aea66487'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:25.312236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90d510673c3d4f038edcbe92aea66487'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:25.324799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90d510673c3d4f038edcbe92aea66487'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:25.330194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:25.332526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90d510673c3d4f038edcbe92aea66487'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:25.346489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1130/2241] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:23:25 2022] DELETE /identity/v3/users/90d510673c3d4f038edcbe92aea66487 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:23:25.351632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1306ef0-23a6-490b-a15e-c3102881a3c6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:25.352012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1306ef0-23a6-490b-a15e-c3102881a3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:25.352293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1306ef0-23a6-490b-a15e-c3102881a3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:25.369885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a1306ef0-23a6-490b-a15e-c3102881a3c6 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:25.371091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a1306ef0-23a6-490b-a15e-c3102881a3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:25.432111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1112/2242] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:23:25.441673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:25.443228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:25.511635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c4c00-8904-4a66-9b22-ac5f1f3cadc0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:25.515578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7094a-ba19-4f9b-a508-6c2b0dccea07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:25.516226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7094a-ba19-4f9b-a508-6c2b0dccea07 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:25.516506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7094a-ba19-4f9b-a508-6c2b0dccea07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:25.516780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7094a-ba19-4f9b-a508-6c2b0dccea07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:25.527078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7094a-ba19-4f9b-a508-6c2b0dccea07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:25.528503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7094a-ba19-4f9b-a508-6c2b0dccea07 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:25.538925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1131/2243] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:25.813737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1306ef0-23a6-490b-a15e-c3102881a3c6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:25.815324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a1306ef0-23a6-490b-a15e-c3102881a3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:25.831760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1306ef0-23a6-490b-a15e-c3102881a3c6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:25.837159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:25.839679 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:25.839968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:25.840247 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] PATH_INFO: `/v3/projects/4641a4d02b7a40c18baa78bdf652332e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:25.841473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4641a4d02b7a40c18baa78bdf652332e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:25.843520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:25.899182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4641a4d02b7a40c18baa78bdf652332e', 'request_id': 'req-1b1222c2-6dfb-4827-a255-3e5cb8413242'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:25.905756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4641a4d02b7a40c18baa78bdf652332e', 'request_id': 'req-1b1222c2-6dfb-4827-a255-3e5cb8413242'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:25.913978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4641a4d02b7a40c18baa78bdf652332e', 'request_id': 'req-1b1222c2-6dfb-4827-a255-3e5cb8413242'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:25.928724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] The token cache is being invalidate because project 4641a4d02b7a40c18baa78bdf652332e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:23:25.929164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:25.929945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b1222c2-6dfb-4827-a255-3e5cb8413242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:25.931981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1113/2244] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:23:25 2022] DELETE /identity/v3/projects/4641a4d02b7a40c18baa78bdf652332e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:23:25.936928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03becc-c252-4009-bc8f-81816e0b70de None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:25.937423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03becc-c252-4009-bc8f-81816e0b70de None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:25.937605 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03becc-c252-4009-bc8f-81816e0b70de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:25.957039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-be03becc-c252-4009-bc8f-81816e0b70de None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:25.958360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-be03becc-c252-4009-bc8f-81816e0b70de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:26.026256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1132/2245] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:23:26.033747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683e51-eec7-41ae-acdc-e15e500c1be7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:26.034046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683e51-eec7-41ae-acdc-e15e500c1be7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:26.034328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683e51-eec7-41ae-acdc-e15e500c1be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:26.052863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ff683e51-eec7-41ae-acdc-e15e500c1be7 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:26.054755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ff683e51-eec7-41ae-acdc-e15e500c1be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:26.062615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1114/2246] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:23:26.066634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03becc-c252-4009-bc8f-81816e0b70de None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:26.076187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03becc-c252-4009-bc8f-81816e0b70de None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:26.080719 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af951a-f59d-48cf-a110-a99a9ffad7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:26.081370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af951a-f59d-48cf-a110-a99a9ffad7f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:26.081653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af951a-f59d-48cf-a110-a99a9ffad7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:26.082027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af951a-f59d-48cf-a110-a99a9ffad7f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:26.082754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af951a-f59d-48cf-a110-a99a9ffad7f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:23:26.083062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af951a-f59d-48cf-a110-a99a9ffad7f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:26.084647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af951a-f59d-48cf-a110-a99a9ffad7f9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:26.095957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1133/2247] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:23:26.100314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff683e51-eec7-41ae-acdc-e15e500c1be7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:26.110355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff683e51-eec7-41ae-acdc-e15e500c1be7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:26.115288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc161a-202d-4710-a690-b1d0d212b8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:26.115933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc161a-202d-4710-a690-b1d0d212b8db admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:26.116221 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc161a-202d-4710-a690-b1d0d212b8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:26.117153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc161a-202d-4710-a690-b1d0d212b8db admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:26.117971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc161a-202d-4710-a690-b1d0d212b8db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:26.119873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc161a-202d-4710-a690-b1d0d212b8db admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:26.146549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1115/2248] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:23:26 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:23:26.150869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af951a-f59d-48cf-a110-a99a9ffad7f9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:26.161232 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af951a-f59d-48cf-a110-a99a9ffad7f9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:26.217581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92f48e-baed-4433-9d05-a5cd658edd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:26.218290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92f48e-baed-4433-9d05-a5cd658edd63 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:26.218574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92f48e-baed-4433-9d05-a5cd658edd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:26.218851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92f48e-baed-4433-9d05-a5cd658edd63 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:26.219554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92f48e-baed-4433-9d05-a5cd658edd63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:26.221519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92f48e-baed-4433-9d05-a5cd658edd63 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:26.243126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1134/2249] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:23:26 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:23:26.247567 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc161a-202d-4710-a690-b1d0d212b8db admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:26.258481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc161a-202d-4710-a690-b1d0d212b8db admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:26.263513 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a038670-2392-43c5-8094-73ec4f8c7f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:26.264252 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a038670-2392-43c5-8094-73ec4f8c7f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:26.264535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a038670-2392-43c5-8094-73ec4f8c7f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:26.264812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a038670-2392-43c5-8094-73ec4f8c7f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:26.265538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a038670-2392-43c5-8094-73ec4f8c7f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:26.267205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a038670-2392-43c5-8094-73ec4f8c7f44 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:26.278296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1116/2250] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:23:26.282690 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92f48e-baed-4433-9d05-a5cd658edd63 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:26.293439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92f48e-baed-4433-9d05-a5cd658edd63 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:26.298184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903e436-2dc8-4161-a23f-a2e520325a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:26.298904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903e436-2dc8-4161-a23f-a2e520325a3c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:26.299246 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903e436-2dc8-4161-a23f-a2e520325a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:26.299518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903e436-2dc8-4161-a23f-a2e520325a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:26.300250 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903e436-2dc8-4161-a23f-a2e520325a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:26.301885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903e436-2dc8-4161-a23f-a2e520325a3c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:26.304506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:26.314171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1135/2251] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:23:26.318271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a038670-2392-43c5-8094-73ec4f8c7f44 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:26.329082 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a038670-2392-43c5-8094-73ec4f8c7f44 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:26.333860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a82636-5861-48ab-a986-25e0d262c2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:26.334799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82636-5861-48ab-a986-25e0d262c2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:26.335083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82636-5861-48ab-a986-25e0d262c2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:26.335361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82636-5861-48ab-a986-25e0d262c2bb admin admin] PATH_INFO: `/v3/projects/d56496025ff74432beb2919cc1784aff/users/3b800fc333314ababf3f54c684fcc967/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:26.350209 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a82636-5861-48ab-a986-25e0d262c2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56496025ff74432beb2919cc1784aff, user_id=3b800fc333314ababf3f54c684fcc967, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:26.352942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a82636-5861-48ab-a986-25e0d262c2bb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:26.367518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1117/2252] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:23:26 2022] PUT /identity/v3/projects/d56496025ff74432beb2919cc1784aff/users/3b800fc333314ababf3f54c684fcc967/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:23:26.379292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7baf3c-6ce7-4a55-84e6-c5aae3aca20d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:26.379586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7baf3c-6ce7-4a55-84e6-c5aae3aca20d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:26.379867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7baf3c-6ce7-4a55-84e6-c5aae3aca20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:26.406755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1c7baf3c-6ce7-4a55-84e6-c5aae3aca20d None None] MFA Rules not processed for user `3b800fc333314ababf3f54c684fcc967`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:26.407881 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1c7baf3c-6ce7-4a55-84e6-c5aae3aca20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:26.484115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1136/2253] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:23:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:23:26.496880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a82636-5861-48ab-a986-25e0d262c2bb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:26.498677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-97a82636-5861-48ab-a986-25e0d262c2bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:26.518219 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:26.560130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a82636-5861-48ab-a986-25e0d262c2bb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:26.565228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e3f25-31a3-469c-9e23-170aaad0ad3b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8386ca93fe48460ba5b6f0c10dc51122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:26.565926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e3f25-31a3-469c-9e23-170aaad0ad3b service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:26.566217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e3f25-31a3-469c-9e23-170aaad0ad3b service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:26.566499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e3f25-31a3-469c-9e23-170aaad0ad3b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:26.576207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e3f25-31a3-469c-9e23-170aaad0ad3b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:26.577642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e3f25-31a3-469c-9e23-170aaad0ad3b service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:26.586745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1118/2254] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:23:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:27.997283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf7636-bed7-4308-ac69-60abe699e568 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:27.998007 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf7636-bed7-4308-ac69-60abe699e568 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:27.998007 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf7636-bed7-4308-ac69-60abe699e568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:28.016132 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5ddf7636-bed7-4308-ac69-60abe699e568 None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:23:28.017339 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5ddf7636-bed7-4308-ac69-60abe699e568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:28.079411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1137/2255] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:23:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:23:28.091789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e3f25-31a3-469c-9e23-170aaad0ad3b service glance] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:28.093264 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d27e3f25-31a3-469c-9e23-170aaad0ad3b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:28.171217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e3f25-31a3-469c-9e23-170aaad0ad3b service glance] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:28.176147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68c1e6-bcb7-43ab-806b-45313764fcb1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:28.176792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68c1e6-bcb7-43ab-806b-45313764fcb1 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:28.177075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68c1e6-bcb7-43ab-806b-45313764fcb1 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:28.177356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68c1e6-bcb7-43ab-806b-45313764fcb1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:28.186746 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68c1e6-bcb7-43ab-806b-45313764fcb1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:28.188326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68c1e6-bcb7-43ab-806b-45313764fcb1 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:28.196943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1119/2256] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:23:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:23:38.067389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=58 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:38.193048 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=51 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:24:01.865863 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33ccab-283f-4fa2-85a1-ded3b26ad61a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:01.866377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33ccab-283f-4fa2-85a1-ded3b26ad61a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:01.866489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33ccab-283f-4fa2-85a1-ded3b26ad61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:01.884502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-df33ccab-283f-4fa2-85a1-ded3b26ad61a None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:24:01.886086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-df33ccab-283f-4fa2-85a1-ded3b26ad61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:01.893190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1138/2257] 10.222.0.42 () {58 vars in 967 bytes} [Wed May 25 16:24:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:24:01.905627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68c1e6-bcb7-43ab-806b-45313764fcb1 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:01.916307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68c1e6-bcb7-43ab-806b-45313764fcb1 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:01.922135 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81fa7d-6562-43b6-9d43-55a599361927 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:01.922769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81fa7d-6562-43b6-9d43-55a599361927 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:01.923054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81fa7d-6562-43b6-9d43-55a599361927 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:01.923329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81fa7d-6562-43b6-9d43-55a599361927 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:01.936993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81fa7d-6562-43b6-9d43-55a599361927 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:01.938872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81fa7d-6562-43b6-9d43-55a599361927 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:01.947420 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1120/2258] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:24:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:24:05.094485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33ccab-283f-4fa2-85a1-ded3b26ad61a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:05.155497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33ccab-283f-4fa2-85a1-ded3b26ad61a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:05.159991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:05.160851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:05.161133 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:05.161408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] PATH_INFO: `/v3/users/3b800fc333314ababf3f54c684fcc967` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:05.162855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b800fc333314ababf3f54c684fcc967)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:05.164773 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:05.211020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b800fc333314ababf3f54c684fcc967'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:05.211676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b800fc333314ababf3f54c684fcc967'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:05.212348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b800fc333314ababf3f54c684fcc967'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:05.224645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b800fc333314ababf3f54c684fcc967'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:05.231361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b800fc333314ababf3f54c684fcc967'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:05.244421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1139/2259] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:24:05 2022] DELETE /identity/v3/users/3b800fc333314ababf3f54c684fcc967 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:24:05.249413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c629a6-b277-4d8e-9bdf-b2d165980e36 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:05.249982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c629a6-b277-4d8e-9bdf-b2d165980e36 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:05.249982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c629a6-b277-4d8e-9bdf-b2d165980e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:05.266608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-61c629a6-b277-4d8e-9bdf-b2d165980e36 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:24:05.267790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-61c629a6-b277-4d8e-9bdf-b2d165980e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:05.338286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1121/2260] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:24:05.348763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:05.350241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:05.407380 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e9a7e-f3b0-4f45-976c-938b51f25a7f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:05.411065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec141ab-da81-4995-8e39-8034751edc06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:05.411705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec141ab-da81-4995-8e39-8034751edc06 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:05.411991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec141ab-da81-4995-8e39-8034751edc06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:05.412524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec141ab-da81-4995-8e39-8034751edc06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:05.422071 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec141ab-da81-4995-8e39-8034751edc06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:05.423508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec141ab-da81-4995-8e39-8034751edc06 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:05.433169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1140/2261] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:24:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:24:05.694181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c629a6-b277-4d8e-9bdf-b2d165980e36 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:05.695901 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-61c629a6-b277-4d8e-9bdf-b2d165980e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:05.707942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c629a6-b277-4d8e-9bdf-b2d165980e36 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:05.712571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:05.713437 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:05.713734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:05.714022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] PATH_INFO: `/v3/projects/d56496025ff74432beb2919cc1784aff` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:05.715211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56496025ff74432beb2919cc1784aff)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:05.717386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:05.766321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56496025ff74432beb2919cc1784aff', 'request_id': 'req-63b59240-7618-484f-b334-aee5cedb582e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:05.771722 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56496025ff74432beb2919cc1784aff', 'request_id': 'req-63b59240-7618-484f-b334-aee5cedb582e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:05.777741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56496025ff74432beb2919cc1784aff', 'request_id': 'req-63b59240-7618-484f-b334-aee5cedb582e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:05.790000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] The token cache is being invalidate because project d56496025ff74432beb2919cc1784aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:24:05.790305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:05.790927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63b59240-7618-484f-b334-aee5cedb582e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:05.792465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1122/2262] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:24:05 2022] DELETE /identity/v3/projects/d56496025ff74432beb2919cc1784aff => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:24:05.798403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f19df-4394-4a3f-b749-df11771604e1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:05.798694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f19df-4394-4a3f-b749-df11771604e1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:05.798978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f19df-4394-4a3f-b749-df11771604e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:05.816896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3f7f19df-4394-4a3f-b749-df11771604e1 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:24:05.818159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3f7f19df-4394-4a3f-b749-df11771604e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:05.827763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:24:05.879273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1141/2263] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:24:05.887138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f473d-d705-474f-ab94-3b5f9cb9ff36 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:05.887431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f473d-d705-474f-ab94-3b5f9cb9ff36 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:05.887732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f473d-d705-474f-ab94-3b5f9cb9ff36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:05.906250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ab6f473d-d705-474f-ab94-3b5f9cb9ff36 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:24:05.907481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ab6f473d-d705-474f-ab94-3b5f9cb9ff36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:05.915815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1123/2264] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:24:05.920502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f19df-4394-4a3f-b749-df11771604e1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:05.930019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f19df-4394-4a3f-b749-df11771604e1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:05.934509 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edef7c6-e30e-476d-8fdd-b4999396b34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:05.935147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef7c6-e30e-476d-8fdd-b4999396b34a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:05.935490 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef7c6-e30e-476d-8fdd-b4999396b34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:05.935827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef7c6-e30e-476d-8fdd-b4999396b34a admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:05.936551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edef7c6-e30e-476d-8fdd-b4999396b34a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:24:05.936855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edef7c6-e30e-476d-8fdd-b4999396b34a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:05.938492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edef7c6-e30e-476d-8fdd-b4999396b34a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:05.949650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1142/2265] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:24:05.954036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f473d-d705-474f-ab94-3b5f9cb9ff36 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:05.963973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f473d-d705-474f-ab94-3b5f9cb9ff36 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:05.969131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ea33-5d7b-4930-b2f8-0a3dc92088e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:05.969825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ea33-5d7b-4930-b2f8-0a3dc92088e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:05.970110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ea33-5d7b-4930-b2f8-0a3dc92088e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:05.970411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ea33-5d7b-4930-b2f8-0a3dc92088e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:05.971129 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050ea33-5d7b-4930-b2f8-0a3dc92088e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:05.973028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050ea33-5d7b-4930-b2f8-0a3dc92088e7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:05.998720 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1124/2266] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:24:05 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:24:06.002823 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edef7c6-e30e-476d-8fdd-b4999396b34a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:06.012544 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edef7c6-e30e-476d-8fdd-b4999396b34a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:06.068830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce0352-92cb-4eea-817f-19a19a08f086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:06.069487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce0352-92cb-4eea-817f-19a19a08f086 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:06.069804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce0352-92cb-4eea-817f-19a19a08f086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:06.070085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce0352-92cb-4eea-817f-19a19a08f086 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:06.070775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce0352-92cb-4eea-817f-19a19a08f086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:06.072704 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce0352-92cb-4eea-817f-19a19a08f086 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:06.092382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1143/2267] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:24:06 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:24:06.096815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ea33-5d7b-4930-b2f8-0a3dc92088e7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:06.107199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ea33-5d7b-4930-b2f8-0a3dc92088e7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:06.112405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23da91-cc2c-4cc6-9c5d-20a6a9d42afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:06.113061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23da91-cc2c-4cc6-9c5d-20a6a9d42afa admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:06.113348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23da91-cc2c-4cc6-9c5d-20a6a9d42afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:06.113627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23da91-cc2c-4cc6-9c5d-20a6a9d42afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:06.114385 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23da91-cc2c-4cc6-9c5d-20a6a9d42afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:06.115981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23da91-cc2c-4cc6-9c5d-20a6a9d42afa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:06.126458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1125/2268] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:24:06.130233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce0352-92cb-4eea-817f-19a19a08f086 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:06.140305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce0352-92cb-4eea-817f-19a19a08f086 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:06.144912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a09f4c-ccb4-4dca-93c0-07d367721f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:06.145557 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09f4c-ccb4-4dca-93c0-07d367721f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:06.145888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09f4c-ccb4-4dca-93c0-07d367721f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:06.146164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09f4c-ccb4-4dca-93c0-07d367721f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:06.146917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a09f4c-ccb4-4dca-93c0-07d367721f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:06.148504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a09f4c-ccb4-4dca-93c0-07d367721f32 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:06.158791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1144/2269] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:24:06.162873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23da91-cc2c-4cc6-9c5d-20a6a9d42afa admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:06.173369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23da91-cc2c-4cc6-9c5d-20a6a9d42afa admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:06.178034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afd959-7eea-4699-96d6-78afbaa0da46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:06.178904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd959-7eea-4699-96d6-78afbaa0da46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:06.179187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd959-7eea-4699-96d6-78afbaa0da46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:06.179465 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd959-7eea-4699-96d6-78afbaa0da46 admin admin] PATH_INFO: `/v3/projects/0cbd73c6967c4e7f9f323ccb00a3279d/users/62a3eaa0fea4459ea84666c6864fda61/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:06.193917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afd959-7eea-4699-96d6-78afbaa0da46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cbd73c6967c4e7f9f323ccb00a3279d, user_id=62a3eaa0fea4459ea84666c6864fda61, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:06.196572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afd959-7eea-4699-96d6-78afbaa0da46 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:06.209457 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1126/2270] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:24:06 2022] PUT /identity/v3/projects/0cbd73c6967c4e7f9f323ccb00a3279d/users/62a3eaa0fea4459ea84666c6864fda61/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:24:06.221424 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87878863-0395-4d85-ac83-4bbf7fe81050 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:06.221736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87878863-0395-4d85-ac83-4bbf7fe81050 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:06.222026 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87878863-0395-4d85-ac83-4bbf7fe81050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:06.248425 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-87878863-0395-4d85-ac83-4bbf7fe81050 None None] MFA Rules not processed for user `62a3eaa0fea4459ea84666c6864fda61`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:24:06.249650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-87878863-0395-4d85-ac83-4bbf7fe81050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:06.322123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1145/2271] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:24:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:24:06.334015 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afd959-7eea-4699-96d6-78afbaa0da46 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:06.335805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-63afd959-7eea-4699-96d6-78afbaa0da46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:06.347838 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:24:06.396605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afd959-7eea-4699-96d6-78afbaa0da46 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:06.400140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffc5df-2546-4559-999a-c5574a35099d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:06.400777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc5df-2546-4559-999a-c5574a35099d service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:06.401061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc5df-2546-4559-999a-c5574a35099d service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:06.401337 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc5df-2546-4559-999a-c5574a35099d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:06.410520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffc5df-2546-4559-999a-c5574a35099d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:06.411945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffc5df-2546-4559-999a-c5574a35099d service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:06.420865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1127/2272] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:24:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:24:14.447976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87878863-0395-4d85-ac83-4bbf7fe81050 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:14.460478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:24:14.511228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87878863-0395-4d85-ac83-4bbf7fe81050 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:14.515936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:14.516808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:14.517091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:14.517367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] PATH_INFO: `/v3/users/62a3eaa0fea4459ea84666c6864fda61` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:14.518597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a3eaa0fea4459ea84666c6864fda61)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:14.520528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:14.567817 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a3eaa0fea4459ea84666c6864fda61'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:14.568479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a3eaa0fea4459ea84666c6864fda61'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:14.569194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a3eaa0fea4459ea84666c6864fda61'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:14.580759 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a3eaa0fea4459ea84666c6864fda61'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:14.587408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a3eaa0fea4459ea84666c6864fda61'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:14.600074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1146/2273] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:24:14 2022] DELETE /identity/v3/users/62a3eaa0fea4459ea84666c6864fda61 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:24:14.604548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797604f-3f17-4cdd-86cd-560c2e2dc0ce None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:14.605038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797604f-3f17-4cdd-86cd-560c2e2dc0ce None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:14.605151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797604f-3f17-4cdd-86cd-560c2e2dc0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:14.625895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b797604f-3f17-4cdd-86cd-560c2e2dc0ce None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:24:14.627089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b797604f-3f17-4cdd-86cd-560c2e2dc0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:14.691805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1128/2274] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:24:14.703528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:14.705190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:14.773882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f9760-8470-410f-a4d0-d156f1f26b14 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:14.777505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ba8fc-16cc-4ba8-8854-7c7c327c7c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:14.778184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ba8fc-16cc-4ba8-8854-7c7c327c7c33 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:14.778467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ba8fc-16cc-4ba8-8854-7c7c327c7c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:14.778743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ba8fc-16cc-4ba8-8854-7c7c327c7c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:14.787826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ba8fc-16cc-4ba8-8854-7c7c327c7c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:14.789251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ba8fc-16cc-4ba8-8854-7c7c327c7c33 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:14.799019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1147/2275] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:24:15.060623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797604f-3f17-4cdd-86cd-560c2e2dc0ce None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:24:15.062121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b797604f-3f17-4cdd-86cd-560c2e2dc0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:15.074640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797604f-3f17-4cdd-86cd-560c2e2dc0ce None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:24:15.079896 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:24:15.080837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:15.081125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:15.081412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] PATH_INFO: `/v3/projects/0cbd73c6967c4e7f9f323ccb00a3279d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:15.082620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cbd73c6967c4e7f9f323ccb00a3279d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:24:15.084619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:24:15.131793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cbd73c6967c4e7f9f323ccb00a3279d', 'request_id': 'req-bbdd91f8-0640-4863-b6ca-00c182f77aec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:15.137179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cbd73c6967c4e7f9f323ccb00a3279d', 'request_id': 'req-bbdd91f8-0640-4863-b6ca-00c182f77aec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:15.143254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cbd73c6967c4e7f9f323ccb00a3279d', 'request_id': 'req-bbdd91f8-0640-4863-b6ca-00c182f77aec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:15.155936 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] The token cache is being invalidate because project 0cbd73c6967c4e7f9f323ccb00a3279d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:24:15.156223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:15.156846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbdd91f8-0640-4863-b6ca-00c182f77aec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:24:15.158430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1129/2276] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:24:15 2022] DELETE /identity/v3/projects/0cbd73c6967c4e7f9f323ccb00a3279d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:24:24.794354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=74,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:24:25.152500 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:24:36.476991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c60c9-5490-45d3-8e8e-553e1f3193c1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:36.477560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c60c9-5490-45d3-8e8e-553e1f3193c1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:36.477656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c60c9-5490-45d3-8e8e-553e1f3193c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:36.495338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a68c60c9-5490-45d3-8e8e-553e1f3193c1 None None] MFA Rules not processed for user `b781c4c6cb61439fb781988a7da028ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:24:36.496545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a68c60c9-5490-45d3-8e8e-553e1f3193c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:36.555921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1148/2277] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:24:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:24:46.549220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=25 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:24:51.018495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d87f55-e6fc-4f75-a599-4feee63c5aba None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:24:51.019037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d87f55-e6fc-4f75-a599-4feee63c5aba None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:24:51.019037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d87f55-e6fc-4f75-a599-4feee63c5aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:24:51.036847 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-88d87f55-e6fc-4f75-a599-4feee63c5aba None None] MFA Rules not processed for user `30de794b71b343289491531dfbe37f53`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:24:51.038489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-88d87f55-e6fc-4f75-a599-4feee63c5aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:24:51.094866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1130/2278] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:24:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:25:00.035356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15709e2-bf9c-467e-a940-da012e745762 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:00.035896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15709e2-bf9c-467e-a940-da012e745762 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:00.036011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15709e2-bf9c-467e-a940-da012e745762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:00.054732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e15709e2-bf9c-467e-a940-da012e745762 None None] MFA Rules not processed for user `9178e89a790d48d19b25c4c22bd0ca4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:25:00.055933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e15709e2-bf9c-467e-a940-da012e745762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:00.112536 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1149/2279] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:25:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:25:01.087614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=25 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:25:10.104917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=25 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:25:34.851373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d87f55-e6fc-4f75-a599-4feee63c5aba None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:34.852827 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-88d87f55-e6fc-4f75-a599-4feee63c5aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:34.937457 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d87f55-e6fc-4f75-a599-4feee63c5aba None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:34.942960 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18926d7c-ec6a-43ad-b768-770e5bfa18e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:34.943626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18926d7c-ec6a-43ad-b768-770e5bfa18e5 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:34.943895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18926d7c-ec6a-43ad-b768-770e5bfa18e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:34.944167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18926d7c-ec6a-43ad-b768-770e5bfa18e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:34.945992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18926d7c-ec6a-43ad-b768-770e5bfa18e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:34.954165 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15709e2-bf9c-467e-a940-da012e745762 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:34.955656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e15709e2-bf9c-467e-a940-da012e745762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:34.957286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18926d7c-ec6a-43ad-b768-770e5bfa18e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:34.958900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18926d7c-ec6a-43ad-b768-770e5bfa18e5 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:34.968137 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1131/2280] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:25:35.015780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15709e2-bf9c-467e-a940-da012e745762 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:35.019397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76fa3b-5430-45d3-9ff3-fe5a4d0b5911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:35.020049 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76fa3b-5430-45d3-9ff3-fe5a4d0b5911 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:35.020332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76fa3b-5430-45d3-9ff3-fe5a4d0b5911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:35.020615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76fa3b-5430-45d3-9ff3-fe5a4d0b5911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:35.022426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1f76fa3b-5430-45d3-9ff3-fe5a4d0b5911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:35.037058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76fa3b-5430-45d3-9ff3-fe5a4d0b5911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:35.038540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76fa3b-5430-45d3-9ff3-fe5a4d0b5911 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:35.050135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1150/2281] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:25:36.225903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18926d7c-ec6a-43ad-b768-770e5bfa18e5 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:36.236783 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18926d7c-ec6a-43ad-b768-770e5bfa18e5 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:36.240957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f42511-77aa-4144-9c98-a5c60fcd7349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:36.241595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f42511-77aa-4144-9c98-a5c60fcd7349 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:36.241908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f42511-77aa-4144-9c98-a5c60fcd7349 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:36.242186 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f42511-77aa-4144-9c98-a5c60fcd7349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:36.243943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b0f42511-77aa-4144-9c98-a5c60fcd7349 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:36.258602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f42511-77aa-4144-9c98-a5c60fcd7349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:36.260041 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f42511-77aa-4144-9c98-a5c60fcd7349 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:36.273968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1132/2282] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:25:39.708959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76fa3b-5430-45d3-9ff3-fe5a4d0b5911 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:39.720868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76fa3b-5430-45d3-9ff3-fe5a4d0b5911 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:39.726300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987083e-5b3f-4b5a-91b4-07ab1c673178 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:39.726987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987083e-5b3f-4b5a-91b4-07ab1c673178 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:39.727270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987083e-5b3f-4b5a-91b4-07ab1c673178 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:39.727548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987083e-5b3f-4b5a-91b4-07ab1c673178 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:39.729277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3987083e-5b3f-4b5a-91b4-07ab1c673178 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:39.786610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987083e-5b3f-4b5a-91b4-07ab1c673178 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:39.788467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987083e-5b3f-4b5a-91b4-07ab1c673178 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:39.797457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1151/2283] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:25:41.688246 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f42511-77aa-4144-9c98-a5c60fcd7349 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:41.689852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b0f42511-77aa-4144-9c98-a5c60fcd7349 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:41.703592 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f42511-77aa-4144-9c98-a5c60fcd7349 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:41.710205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b59841-da79-40a8-8502-a1113a809bfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:41.710900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59841-da79-40a8-8502-a1113a809bfe service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:41.711326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59841-da79-40a8-8502-a1113a809bfe service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:41.711645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59841-da79-40a8-8502-a1113a809bfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:41.713549 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-54b59841-da79-40a8-8502-a1113a809bfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:41.724738 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b59841-da79-40a8-8502-a1113a809bfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:41.726402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b59841-da79-40a8-8502-a1113a809bfe service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:41.736069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1133/2284] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:25:41.827125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1da9f-7c08-4c52-aec7-c2e775fc7b9b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:41.827635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1da9f-7c08-4c52-aec7-c2e775fc7b9b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:41.827739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1da9f-7c08-4c52-aec7-c2e775fc7b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:41.844833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dea1da9f-7c08-4c52-aec7-c2e775fc7b9b None None] MFA Rules not processed for user `15fae427f22d476fa0e378949de41292`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:25:41.846071 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dea1da9f-7c08-4c52-aec7-c2e775fc7b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:41.852886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1152/2285] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:25:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:25:42.151588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b59841-da79-40a8-8502-a1113a809bfe service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:42.161183 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b59841-da79-40a8-8502-a1113a809bfe service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:42.165028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68e2dc-01fb-49f1-b566-c5cebb1a811b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:42.165663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68e2dc-01fb-49f1-b566-c5cebb1a811b service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:42.165991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68e2dc-01fb-49f1-b566-c5cebb1a811b service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:42.166266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68e2dc-01fb-49f1-b566-c5cebb1a811b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:42.168782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7e68e2dc-01fb-49f1-b566-c5cebb1a811b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:42.178507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68e2dc-01fb-49f1-b566-c5cebb1a811b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:42.179923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68e2dc-01fb-49f1-b566-c5cebb1a811b service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:42.188761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1134/2286] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:25:42.508522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1da9f-7c08-4c52-aec7-c2e775fc7b9b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:42.509929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dea1da9f-7c08-4c52-aec7-c2e775fc7b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:42.597847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1da9f-7c08-4c52-aec7-c2e775fc7b9b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:42.610173 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46ba0c-8b86-4b15-8c57-b536a4930c6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:42.611049 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46ba0c-8b86-4b15-8c57-b536a4930c6a service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:42.611433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46ba0c-8b86-4b15-8c57-b536a4930c6a service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:42.611811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46ba0c-8b86-4b15-8c57-b536a4930c6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:42.613842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2d46ba0c-8b86-4b15-8c57-b536a4930c6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:42.626806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46ba0c-8b86-4b15-8c57-b536a4930c6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:42.628456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46ba0c-8b86-4b15-8c57-b536a4930c6a service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:42.640263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1153/2287] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:25:44.915447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f9ef9-aa4b-4a0f-aac2-d9ce23143be2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:44.915931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f9ef9-aa4b-4a0f-aac2-d9ce23143be2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:44.916059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f9ef9-aa4b-4a0f-aac2-d9ce23143be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:44.935888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8d1f9ef9-aa4b-4a0f-aac2-d9ce23143be2 None None] MFA Rules not processed for user `c935f035c757482ca912a8c4b77714a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:25:44.937099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8d1f9ef9-aa4b-4a0f-aac2-d9ce23143be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:45.021102 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1135/2288] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:25:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:25:45.358001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46ba0c-8b86-4b15-8c57-b536a4930c6a service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:45.367585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46ba0c-8b86-4b15-8c57-b536a4930c6a service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:45.372267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53732e-523f-4767-8233-3361f55836e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:45.373087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53732e-523f-4767-8233-3361f55836e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:45.373218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53732e-523f-4767-8233-3361f55836e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:45.373488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53732e-523f-4767-8233-3361f55836e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:45.375307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-af53732e-523f-4767-8233-3361f55836e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:45.385180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53732e-523f-4767-8233-3361f55836e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:45.386910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53732e-523f-4767-8233-3361f55836e6 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:45.396409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1154/2289] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:25:45.819772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f9ef9-aa4b-4a0f-aac2-d9ce23143be2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:45.821231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8d1f9ef9-aa4b-4a0f-aac2-d9ce23143be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:45.847723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f9ef9-aa4b-4a0f-aac2-d9ce23143be2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:45.860803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4977-c8d4-415c-82fd-b4835e0f5abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:45.862253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4977-c8d4-415c-82fd-b4835e0f5abe service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:45.862816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4977-c8d4-415c-82fd-b4835e0f5abe service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:45.863835 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4977-c8d4-415c-82fd-b4835e0f5abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:45.868225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-193c4977-c8d4-415c-82fd-b4835e0f5abe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:45.883214 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c4977-c8d4-415c-82fd-b4835e0f5abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:45.884969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c4977-c8d4-415c-82fd-b4835e0f5abe service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:45.897478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1136/2290] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:25:52.007822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53732e-523f-4767-8233-3361f55836e6 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:52.024534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53732e-523f-4767-8233-3361f55836e6 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:52.031022 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6bb1c-5054-4a1a-963e-3809bdae8449 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:52.031682 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bb1c-5054-4a1a-963e-3809bdae8449 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:52.031970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bb1c-5054-4a1a-963e-3809bdae8449 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:52.032252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bb1c-5054-4a1a-963e-3809bdae8449 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:52.034129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e7e6bb1c-5054-4a1a-963e-3809bdae8449 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:52.048043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6bb1c-5054-4a1a-963e-3809bdae8449 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:52.049655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6bb1c-5054-4a1a-963e-3809bdae8449 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:52.054745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:25:52.062503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1155/2291] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:25:55.893553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=83 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:25:57.638884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4977-c8d4-415c-82fd-b4835e0f5abe service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:57.651505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4977-c8d4-415c-82fd-b4835e0f5abe service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:57.658127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6836e3f-b78c-46e0-af92-acb4a13d02b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:57.658768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6836e3f-b78c-46e0-af92-acb4a13d02b6 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:57.659050 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6836e3f-b78c-46e0-af92-acb4a13d02b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:57.659325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6836e3f-b78c-46e0-af92-acb4a13d02b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:57.661131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a6836e3f-b78c-46e0-af92-acb4a13d02b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:57.674414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6836e3f-b78c-46e0-af92-acb4a13d02b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:57.675978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6836e3f-b78c-46e0-af92-acb4a13d02b6 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:57.686206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1137/2292] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:25:58.354134 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6bb1c-5054-4a1a-963e-3809bdae8449 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:25:58.367174 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6bb1c-5054-4a1a-963e-3809bdae8449 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:25:58.373499 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139a4f1-2ac1-49f2-a346-e239a11fba91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:25:58.374204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139a4f1-2ac1-49f2-a346-e239a11fba91 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:25:58.374604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139a4f1-2ac1-49f2-a346-e239a11fba91 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:25:58.374893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139a4f1-2ac1-49f2-a346-e239a11fba91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:25:58.376738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6139a4f1-2ac1-49f2-a346-e239a11fba91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:25:58.390535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139a4f1-2ac1-49f2-a346-e239a11fba91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:25:58.392236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139a4f1-2ac1-49f2-a346-e239a11fba91 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:25:58.403245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1156/2293] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:25:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:26:00.107552 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9e6a0-bf35-42f3-b164-5a1400aae01e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:00.108060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9e6a0-bf35-42f3-b164-5a1400aae01e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:00.108160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9e6a0-bf35-42f3-b164-5a1400aae01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:00.126621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-29b9e6a0-bf35-42f3-b164-5a1400aae01e None None] MFA Rules not processed for user `b781c4c6cb61439fb781988a7da028ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:00.127800 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-29b9e6a0-bf35-42f3-b164-5a1400aae01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:00.134589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1138/2294] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:00.407452 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139a4f1-2ac1-49f2-a346-e239a11fba91 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:00.420201 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139a4f1-2ac1-49f2-a346-e239a11fba91 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:00.430520 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99374b2-d49e-41e2-8373-c3758be8da86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:00.431194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99374b2-d49e-41e2-8373-c3758be8da86 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:00.431478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99374b2-d49e-41e2-8373-c3758be8da86 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:00.431754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99374b2-d49e-41e2-8373-c3758be8da86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:00.436218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e99374b2-d49e-41e2-8373-c3758be8da86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:00.447570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99374b2-d49e-41e2-8373-c3758be8da86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:00.449210 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99374b2-d49e-41e2-8373-c3758be8da86 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:00.459614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1157/2295] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:08.580365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9e6a0-bf35-42f3-b164-5a1400aae01e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:08.592295 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9e6a0-bf35-42f3-b164-5a1400aae01e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:08.597927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec31385-df44-441b-866f-98322cef7358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:08.598713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec31385-df44-441b-866f-98322cef7358 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:08.599103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec31385-df44-441b-866f-98322cef7358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:08.599833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec31385-df44-441b-866f-98322cef7358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:08.602184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cec31385-df44-441b-866f-98322cef7358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:08.678347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec31385-df44-441b-866f-98322cef7358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:08.679597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec31385-df44-441b-866f-98322cef7358 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:08.691043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1139/2296] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:10.455170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=13 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:13.058123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99374b2-d49e-41e2-8373-c3758be8da86 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:13.061574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e99374b2-d49e-41e2-8373-c3758be8da86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:13.079036 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99374b2-d49e-41e2-8373-c3758be8da86 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:13.083877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8d498-0bff-4fce-b1e9-fa035cb09d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:13.084827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8d498-0bff-4fce-b1e9-fa035cb09d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:13.085184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8d498-0bff-4fce-b1e9-fa035cb09d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:13.085466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8d498-0bff-4fce-b1e9-fa035cb09d0c admin admin] PATH_INFO: `/v3/users/30de794b71b343289491531dfbe37f53` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:13.086713 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8d498-0bff-4fce-b1e9-fa035cb09d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=30de794b71b343289491531dfbe37f53)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:13.088652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8d498-0bff-4fce-b1e9-fa035cb09d0c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:13.146203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4b8d498-0bff-4fce-b1e9-fa035cb09d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30de794b71b343289491531dfbe37f53'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:13.146873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4b8d498-0bff-4fce-b1e9-fa035cb09d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30de794b71b343289491531dfbe37f53'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:13.147496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4b8d498-0bff-4fce-b1e9-fa035cb09d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30de794b71b343289491531dfbe37f53'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:13.160856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4b8d498-0bff-4fce-b1e9-fa035cb09d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30de794b71b343289491531dfbe37f53'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:13.168676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4b8d498-0bff-4fce-b1e9-fa035cb09d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30de794b71b343289491531dfbe37f53'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:13.181798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1158/2297] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:26:13 2022] DELETE /identity/v3/users/30de794b71b343289491531dfbe37f53 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:13.376168 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec31385-df44-441b-866f-98322cef7358 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:13.400519 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec31385-df44-441b-866f-98322cef7358 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:13.484665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:13.486026 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:13.486429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:13.486906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] PATH_INFO: `/v3/projects/95d1e25b20a146c198dfeda0dc43cd72` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:13.488627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d1e25b20a146c198dfeda0dc43cd72)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:13.490925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:13.586816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d1e25b20a146c198dfeda0dc43cd72', 'request_id': 'req-744a90a0-a5f5-43db-9cf7-b263b3c199c8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:13.597795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d1e25b20a146c198dfeda0dc43cd72', 'request_id': 'req-744a90a0-a5f5-43db-9cf7-b263b3c199c8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:13.606027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d1e25b20a146c198dfeda0dc43cd72', 'request_id': 'req-744a90a0-a5f5-43db-9cf7-b263b3c199c8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:13.622777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] The token cache is being invalidate because project 95d1e25b20a146c198dfeda0dc43cd72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:13.623343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:13.624272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-744a90a0-a5f5-43db-9cf7-b263b3c199c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:13.629557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1140/2298] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:13 2022] DELETE /identity/v3/projects/95d1e25b20a146c198dfeda0dc43cd72 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:13.631250 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add685aa-f830-4ad0-b090-f968166d91b8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:13.631565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add685aa-f830-4ad0-b090-f968166d91b8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:13.631907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add685aa-f830-4ad0-b090-f968166d91b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:13.652569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-add685aa-f830-4ad0-b090-f968166d91b8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:13.653808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-add685aa-f830-4ad0-b090-f968166d91b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:13.717299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1159/2299] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:26:13.724663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fa169-5e0b-4cc6-8e5a-8ea5946ee9db None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:13.724951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fa169-5e0b-4cc6-8e5a-8ea5946ee9db None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:13.725222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fa169-5e0b-4cc6-8e5a-8ea5946ee9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:13.734164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:13.747404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-256fa169-5e0b-4cc6-8e5a-8ea5946ee9db None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:13.749677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-256fa169-5e0b-4cc6-8e5a-8ea5946ee9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:13.761796 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1141/2300] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:13.763656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add685aa-f830-4ad0-b090-f968166d91b8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:13.777278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add685aa-f830-4ad0-b090-f968166d91b8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:13.782827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295e572-44f7-4080-a156-4b3957562e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:13.783612 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295e572-44f7-4080-a156-4b3957562e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:13.784029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295e572-44f7-4080-a156-4b3957562e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:13.784459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295e572-44f7-4080-a156-4b3957562e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:13.785630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295e572-44f7-4080-a156-4b3957562e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:26:13.785848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295e572-44f7-4080-a156-4b3957562e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:13.788157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295e572-44f7-4080-a156-4b3957562e8b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:13.803259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1160/2301] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:26:13.807398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fa169-5e0b-4cc6-8e5a-8ea5946ee9db None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:13.817874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fa169-5e0b-4cc6-8e5a-8ea5946ee9db None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:13.823060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd64e74-b51d-4794-9fa1-7a63ce755a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:13.824126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd64e74-b51d-4794-9fa1-7a63ce755a43 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:13.824507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd64e74-b51d-4794-9fa1-7a63ce755a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:13.824858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd64e74-b51d-4794-9fa1-7a63ce755a43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:13.825635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd64e74-b51d-4794-9fa1-7a63ce755a43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:13.828044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd64e74-b51d-4794-9fa1-7a63ce755a43 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:13.861749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1142/2302] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:26:13 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:26:13.864498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295e572-44f7-4080-a156-4b3957562e8b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:13.876717 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295e572-44f7-4080-a156-4b3957562e8b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:13.936307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c5ea5-8fd4-4e8e-8acb-2825246570c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:13.937066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5ea5-8fd4-4e8e-8acb-2825246570c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:13.937351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5ea5-8fd4-4e8e-8acb-2825246570c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:13.938125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5ea5-8fd4-4e8e-8acb-2825246570c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:13.938347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c5ea5-8fd4-4e8e-8acb-2825246570c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:13.940272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c5ea5-8fd4-4e8e-8acb-2825246570c0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:13.946990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:13.963585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1161/2303] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:26:13 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:13.967977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd64e74-b51d-4794-9fa1-7a63ce755a43 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:13.978756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd64e74-b51d-4794-9fa1-7a63ce755a43 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:13.985093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d5c57-44bf-4397-a8f9-9f0b9be6bca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:13.985932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d5c57-44bf-4397-a8f9-9f0b9be6bca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:13.986210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d5c57-44bf-4397-a8f9-9f0b9be6bca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:13.986505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d5c57-44bf-4397-a8f9-9f0b9be6bca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:13.987295 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d5c57-44bf-4397-a8f9-9f0b9be6bca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:13.989013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d5c57-44bf-4397-a8f9-9f0b9be6bca5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:14.001781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1143/2304] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:14.005676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c5ea5-8fd4-4e8e-8acb-2825246570c0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:14.018853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c5ea5-8fd4-4e8e-8acb-2825246570c0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:14.025404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3da7e-1a79-47f6-82d0-0589f68d44e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:14.027182 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3da7e-1a79-47f6-82d0-0589f68d44e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:14.027792 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3da7e-1a79-47f6-82d0-0589f68d44e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:14.028559 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3da7e-1a79-47f6-82d0-0589f68d44e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:14.029638 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3da7e-1a79-47f6-82d0-0589f68d44e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:14.031624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3da7e-1a79-47f6-82d0-0589f68d44e9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:14.045004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1162/2305] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:14.049761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d5c57-44bf-4397-a8f9-9f0b9be6bca5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:14.060638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d5c57-44bf-4397-a8f9-9f0b9be6bca5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:14.065526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41935d9-6756-42b9-9d4c-1e9ca4e98a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:14.066473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41935d9-6756-42b9-9d4c-1e9ca4e98a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:14.066764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41935d9-6756-42b9-9d4c-1e9ca4e98a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:14.067042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41935d9-6756-42b9-9d4c-1e9ca4e98a16 admin admin] PATH_INFO: `/v3/projects/1976c732674e4a04b03b2edecdb9cbad/users/d4457f8bb0474d2797da7c9307a0be15/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:14.084260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41935d9-6756-42b9-9d4c-1e9ca4e98a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1976c732674e4a04b03b2edecdb9cbad, user_id=d4457f8bb0474d2797da7c9307a0be15, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:14.087030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41935d9-6756-42b9-9d4c-1e9ca4e98a16 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:14.114421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1144/2306] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:14 2022] PUT /identity/v3/projects/1976c732674e4a04b03b2edecdb9cbad/users/d4457f8bb0474d2797da7c9307a0be15/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:14.116334 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559e651-7ddb-416d-b90c-8a393918bcd9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:14.116837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559e651-7ddb-416d-b90c-8a393918bcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:14.116945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559e651-7ddb-416d-b90c-8a393918bcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:14.147972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3559e651-7ddb-416d-b90c-8a393918bcd9 None None] MFA Rules not processed for user `d4457f8bb0474d2797da7c9307a0be15`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:14.149297 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3559e651-7ddb-416d-b90c-8a393918bcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:14.227653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1163/2307] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:14.233066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41935d9-6756-42b9-9d4c-1e9ca4e98a16 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:14.310116 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41935d9-6756-42b9-9d4c-1e9ca4e98a16 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:14.315244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8db48a-373e-4d11-8882-126e7c1c1734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:14.315882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8db48a-373e-4d11-8882-126e7c1c1734 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:14.316184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8db48a-373e-4d11-8882-126e7c1c1734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:14.316482 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8db48a-373e-4d11-8882-126e7c1c1734 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:14.317240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8db48a-373e-4d11-8882-126e7c1c1734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:14.319347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8db48a-373e-4d11-8882-126e7c1c1734 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:14.352379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1145/2308] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:26:14 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:26:14.357180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559e651-7ddb-416d-b90c-8a393918bcd9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:14.368330 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559e651-7ddb-416d-b90c-8a393918bcd9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:14.433851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766626e-5764-42fa-bc9c-584896b19e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:14.434551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766626e-5764-42fa-bc9c-584896b19e10 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:14.434858 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766626e-5764-42fa-bc9c-584896b19e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:14.435569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766626e-5764-42fa-bc9c-584896b19e10 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:14.436330 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766626e-5764-42fa-bc9c-584896b19e10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:14.438363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766626e-5764-42fa-bc9c-584896b19e10 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:14.459402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1164/2309] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:26:14 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:14.463218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8db48a-373e-4d11-8882-126e7c1c1734 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:14.474559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8db48a-373e-4d11-8882-126e7c1c1734 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:14.479254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848c43d-25b5-45f9-b53b-318165134793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:14.480353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c43d-25b5-45f9-b53b-318165134793 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:14.480705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c43d-25b5-45f9-b53b-318165134793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:14.481009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c43d-25b5-45f9-b53b-318165134793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:14.481734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848c43d-25b5-45f9-b53b-318165134793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:14.483342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848c43d-25b5-45f9-b53b-318165134793 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:14.494666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1146/2310] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:26:14.498613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766626e-5764-42fa-bc9c-584896b19e10 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:14.509351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766626e-5764-42fa-bc9c-584896b19e10 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:14.517085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a7c79-daef-46f8-b4d1-e80f143f7774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:14.518013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a7c79-daef-46f8-b4d1-e80f143f7774 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:14.518293 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a7c79-daef-46f8-b4d1-e80f143f7774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:14.518569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a7c79-daef-46f8-b4d1-e80f143f7774 admin admin] PATH_INFO: `/v3/projects/18daa4cd9a61438faa31540c7ba2c18b/users/7da929a5ac3c48e5875d7dbc913e9a77/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:14.538464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a7c79-daef-46f8-b4d1-e80f143f7774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18daa4cd9a61438faa31540c7ba2c18b, user_id=7da929a5ac3c48e5875d7dbc913e9a77, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:14.541117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a7c79-daef-46f8-b4d1-e80f143f7774 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:14.555459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1165/2311] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:14 2022] PUT /identity/v3/projects/18daa4cd9a61438faa31540c7ba2c18b/users/7da929a5ac3c48e5875d7dbc913e9a77/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:14.559908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848c43d-25b5-45f9-b53b-318165134793 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:14.571727 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848c43d-25b5-45f9-b53b-318165134793 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:14.654205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97887e-64f6-4718-a19e-2e6609162d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:14.656022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97887e-64f6-4718-a19e-2e6609162d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:14.656364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97887e-64f6-4718-a19e-2e6609162d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:14.656682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97887e-64f6-4718-a19e-2e6609162d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:14.658146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97887e-64f6-4718-a19e-2e6609162d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:14.659975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97887e-64f6-4718-a19e-2e6609162d68 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:14.675699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1147/2312] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:14.679502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a7c79-daef-46f8-b4d1-e80f143f7774 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:14.692693 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a7c79-daef-46f8-b4d1-e80f143f7774 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:14.697491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3c368-375e-4ee0-9524-977c65582dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:14.699380 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3c368-375e-4ee0-9524-977c65582dba admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:14.699797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3c368-375e-4ee0-9524-977c65582dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:14.700187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3c368-375e-4ee0-9524-977c65582dba admin admin] PATH_INFO: `/v3/projects/18daa4cd9a61438faa31540c7ba2c18b/users/7da929a5ac3c48e5875d7dbc913e9a77/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:14.702387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3c368-375e-4ee0-9524-977c65582dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=18daa4cd9a61438faa31540c7ba2c18b, user_id=7da929a5ac3c48e5875d7dbc913e9a77, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:14.705225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3c368-375e-4ee0-9524-977c65582dba admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:14.719041 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1166/2313] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:14 2022] PUT /identity/v3/projects/18daa4cd9a61438faa31540c7ba2c18b/users/7da929a5ac3c48e5875d7dbc913e9a77/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:14.730821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0632437-cc39-4281-91b8-d6848a78487f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:14.731115 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0632437-cc39-4281-91b8-d6848a78487f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:14.731417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0632437-cc39-4281-91b8-d6848a78487f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:14.760179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:14.771049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e0632437-cc39-4281-91b8-d6848a78487f None None] MFA Rules not processed for user `7da929a5ac3c48e5875d7dbc913e9a77`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:14.773463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e0632437-cc39-4281-91b8-d6848a78487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:14.849534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1148/2314] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:14.857644 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3c368-375e-4ee0-9524-977c65582dba admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:14.859613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-16b3c368-375e-4ee0-9524-977c65582dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:14.896700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:14.939865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3c368-375e-4ee0-9524-977c65582dba admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:14.943928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5067b26-b57e-4d32-a50e-399077e5d1b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:14.944590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5067b26-b57e-4d32-a50e-399077e5d1b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:14.944932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5067b26-b57e-4d32-a50e-399077e5d1b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:14.945296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5067b26-b57e-4d32-a50e-399077e5d1b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:14.956449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5067b26-b57e-4d32-a50e-399077e5d1b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:14.958010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5067b26-b57e-4d32-a50e-399077e5d1b4 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:14.971218 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1167/2315] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:15.060994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0632437-cc39-4281-91b8-d6848a78487f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:15.144897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0632437-cc39-4281-91b8-d6848a78487f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:15.149573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:15.150483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:15.150762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:15.151034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] PATH_INFO: `/v3/users/d4457f8bb0474d2797da7c9307a0be15` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:15.152541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4457f8bb0474d2797da7c9307a0be15)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:15.154501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:15.211032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4457f8bb0474d2797da7c9307a0be15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:15.219731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4457f8bb0474d2797da7c9307a0be15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:15.220358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4457f8bb0474d2797da7c9307a0be15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:15.234822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4457f8bb0474d2797da7c9307a0be15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:15.242361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4457f8bb0474d2797da7c9307a0be15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:15.252362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1149/2316] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:26:15 2022] DELETE /identity/v3/users/d4457f8bb0474d2797da7c9307a0be15 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:15.258062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c73ee-602e-472f-9102-4ef0d960268c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:15.258912 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c73ee-602e-472f-9102-4ef0d960268c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:15.259276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c73ee-602e-472f-9102-4ef0d960268c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:15.279353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dc2c73ee-602e-472f-9102-4ef0d960268c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:15.280658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dc2c73ee-602e-472f-9102-4ef0d960268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:15.347837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1168/2317] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:15.359272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:15.360900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:15.383189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:15.433379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc1f32-8201-4ef0-92ad-713c3ce5cdde admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:15.440874 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e842b-9c1c-479b-a706-bbb9f853196d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:15.441576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e842b-9c1c-479b-a706-bbb9f853196d service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:15.441930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e842b-9c1c-479b-a706-bbb9f853196d service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:15.442225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e842b-9c1c-479b-a706-bbb9f853196d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:15.453021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e842b-9c1c-479b-a706-bbb9f853196d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:15.454568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e842b-9c1c-479b-a706-bbb9f853196d service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:15.465206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1150/2318] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:15.773916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c73ee-602e-472f-9102-4ef0d960268c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:15.775847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dc2c73ee-602e-472f-9102-4ef0d960268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:15.801779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c73ee-602e-472f-9102-4ef0d960268c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:15.812161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:15.813077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:15.813360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:15.813635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] PATH_INFO: `/v3/projects/1976c732674e4a04b03b2edecdb9cbad` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:15.815725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1976c732674e4a04b03b2edecdb9cbad)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:15.817925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:15.879674 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1976c732674e4a04b03b2edecdb9cbad', 'request_id': 'req-2d5a651f-ef69-42b6-9c56-30415bda8f9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:15.888211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1976c732674e4a04b03b2edecdb9cbad', 'request_id': 'req-2d5a651f-ef69-42b6-9c56-30415bda8f9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:15.897373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1976c732674e4a04b03b2edecdb9cbad', 'request_id': 'req-2d5a651f-ef69-42b6-9c56-30415bda8f9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:15.910540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] The token cache is being invalidate because project 1976c732674e4a04b03b2edecdb9cbad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:15.910928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:15.912422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:15.914671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1169/2319] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:15 2022] DELETE /identity/v3/projects/1976c732674e4a04b03b2edecdb9cbad => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:15.918570 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e842b-9c1c-479b-a706-bbb9f853196d service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:15.929425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e842b-9c1c-479b-a706-bbb9f853196d service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:15.994843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e1af8-c83b-4798-a92a-c9d8581a1ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:15.995902 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e1af8-c83b-4798-a92a-c9d8581a1ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:15.996272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e1af8-c83b-4798-a92a-c9d8581a1ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:15.996580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e1af8-c83b-4798-a92a-c9d8581a1ba9 admin admin] PATH_INFO: `/v3/users/7da929a5ac3c48e5875d7dbc913e9a77` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:15.997806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e1af8-c83b-4798-a92a-c9d8581a1ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da929a5ac3c48e5875d7dbc913e9a77)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:15.999768 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e1af8-c83b-4798-a92a-c9d8581a1ba9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:16.051121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-679e1af8-c83b-4798-a92a-c9d8581a1ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da929a5ac3c48e5875d7dbc913e9a77'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:16.058520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-679e1af8-c83b-4798-a92a-c9d8581a1ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da929a5ac3c48e5875d7dbc913e9a77'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:16.059193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-679e1af8-c83b-4798-a92a-c9d8581a1ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da929a5ac3c48e5875d7dbc913e9a77'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:16.072978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-679e1af8-c83b-4798-a92a-c9d8581a1ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da929a5ac3c48e5875d7dbc913e9a77'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:16.080887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-679e1af8-c83b-4798-a92a-c9d8581a1ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da929a5ac3c48e5875d7dbc913e9a77'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:16.086977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1151/2320] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:26:15 2022] DELETE /identity/v3/users/7da929a5ac3c48e5875d7dbc913e9a77 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:16.445402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:16.446963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:16.461526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a651f-ef69-42b6-9c56-30415bda8f9e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:16.513270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:16.528395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:16.529295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:16.529599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:16.529910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] PATH_INFO: `/v3/projects/18daa4cd9a61438faa31540c7ba2c18b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:16.531195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18daa4cd9a61438faa31540c7ba2c18b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:16.533220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:16.582361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18daa4cd9a61438faa31540c7ba2c18b', 'request_id': 'req-2db4f1de-916f-46a4-891a-ebef25751ec3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:16.588341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18daa4cd9a61438faa31540c7ba2c18b', 'request_id': 'req-2db4f1de-916f-46a4-891a-ebef25751ec3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:16.596341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18daa4cd9a61438faa31540c7ba2c18b', 'request_id': 'req-2db4f1de-916f-46a4-891a-ebef25751ec3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:16.608158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] The token cache is being invalidate because project 18daa4cd9a61438faa31540c7ba2c18b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:16.608476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:16.609176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2db4f1de-916f-46a4-891a-ebef25751ec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:16.611011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1170/2321] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:16 2022] DELETE /identity/v3/projects/18daa4cd9a61438faa31540c7ba2c18b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:16.616044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f38d9-ea34-4336-91d4-bc001d6072da None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:16.616588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f38d9-ea34-4336-91d4-bc001d6072da None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:16.616706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f38d9-ea34-4336-91d4-bc001d6072da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:16.636182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e79f38d9-ea34-4336-91d4-bc001d6072da None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:16.638013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e79f38d9-ea34-4336-91d4-bc001d6072da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:16.672982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:16.701661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1152/2322] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:16.709144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f932133-ea26-4fec-8edb-a0a060e679f1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:16.709436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f932133-ea26-4fec-8edb-a0a060e679f1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:16.709728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f932133-ea26-4fec-8edb-a0a060e679f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:16.727581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9f932133-ea26-4fec-8edb-a0a060e679f1 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:16.728786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9f932133-ea26-4fec-8edb-a0a060e679f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:16.737411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1171/2323] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:16.741579 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f38d9-ea34-4336-91d4-bc001d6072da None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:16.752367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f38d9-ea34-4336-91d4-bc001d6072da None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:16.757304 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99469-d85b-4ae1-a1a0-cf689c90113b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:16.758002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99469-d85b-4ae1-a1a0-cf689c90113b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:16.758284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99469-d85b-4ae1-a1a0-cf689c90113b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:16.758559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99469-d85b-4ae1-a1a0-cf689c90113b admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:16.759285 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f99469-d85b-4ae1-a1a0-cf689c90113b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:26:16.759590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f99469-d85b-4ae1-a1a0-cf689c90113b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:16.761526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f99469-d85b-4ae1-a1a0-cf689c90113b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:16.773254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1153/2324] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:26:16.777621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f932133-ea26-4fec-8edb-a0a060e679f1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:16.786934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f932133-ea26-4fec-8edb-a0a060e679f1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:16.791566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b69187-80e7-43e3-943e-bcb86f854244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:16.792749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b69187-80e7-43e3-943e-bcb86f854244 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:16.793028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b69187-80e7-43e3-943e-bcb86f854244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:16.793300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b69187-80e7-43e3-943e-bcb86f854244 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:16.794012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b69187-80e7-43e3-943e-bcb86f854244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:16.795909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b69187-80e7-43e3-943e-bcb86f854244 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:16.823188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1172/2325] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:26:16 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:16.827968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99469-d85b-4ae1-a1a0-cf689c90113b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:16.838667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99469-d85b-4ae1-a1a0-cf689c90113b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:16.894869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc4a40-0b9d-424d-a94a-205788a79d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:16.895545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc4a40-0b9d-424d-a94a-205788a79d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:16.895827 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc4a40-0b9d-424d-a94a-205788a79d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:16.896100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc4a40-0b9d-424d-a94a-205788a79d60 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:16.896781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc4a40-0b9d-424d-a94a-205788a79d60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:16.898801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc4a40-0b9d-424d-a94a-205788a79d60 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:16.918596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1154/2326] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:26:16 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:16.922779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b69187-80e7-43e3-943e-bcb86f854244 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:16.933694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b69187-80e7-43e3-943e-bcb86f854244 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:16.938396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a62dc-c3d6-4b1f-b6a6-ed6704aa1d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:16.939060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a62dc-c3d6-4b1f-b6a6-ed6704aa1d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:16.939351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a62dc-c3d6-4b1f-b6a6-ed6704aa1d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:16.939643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a62dc-c3d6-4b1f-b6a6-ed6704aa1d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:16.940379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a62dc-c3d6-4b1f-b6a6-ed6704aa1d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:16.942113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a62dc-c3d6-4b1f-b6a6-ed6704aa1d47 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:16.953822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1173/2327] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:16.959957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc4a40-0b9d-424d-a94a-205788a79d60 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:16.977107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc4a40-0b9d-424d-a94a-205788a79d60 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:16.985357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00c74a-c304-4321-8398-cb67f2893a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:16.986494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00c74a-c304-4321-8398-cb67f2893a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:16.986961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00c74a-c304-4321-8398-cb67f2893a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:16.987400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00c74a-c304-4321-8398-cb67f2893a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:16.988565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00c74a-c304-4321-8398-cb67f2893a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:16.990930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00c74a-c304-4321-8398-cb67f2893a62 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:17.009475 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1155/2328] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:17.013686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a62dc-c3d6-4b1f-b6a6-ed6704aa1d47 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:17.024306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a62dc-c3d6-4b1f-b6a6-ed6704aa1d47 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:17.031159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fafba-63ef-4496-b308-e361713e8c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:17.032044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fafba-63ef-4496-b308-e361713e8c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:17.032329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fafba-63ef-4496-b308-e361713e8c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:17.032604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fafba-63ef-4496-b308-e361713e8c79 admin admin] PATH_INFO: `/v3/projects/834e969de42342039fe817db706f5043/users/1c1d0d65b960496780853910208e8a9d/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:17.048786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fafba-63ef-4496-b308-e361713e8c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834e969de42342039fe817db706f5043, user_id=1c1d0d65b960496780853910208e8a9d, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:17.051477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fafba-63ef-4496-b308-e361713e8c79 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:17.067274 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1174/2329] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:17 2022] PUT /identity/v3/projects/834e969de42342039fe817db706f5043/users/1c1d0d65b960496780853910208e8a9d/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:26:17.079279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5cfd-4135-4dd6-9352-deb629802de3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:17.079619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5cfd-4135-4dd6-9352-deb629802de3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:17.079972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5cfd-4135-4dd6-9352-deb629802de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:17.111891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-271f5cfd-4135-4dd6-9352-deb629802de3 None None] MFA Rules not processed for user `1c1d0d65b960496780853910208e8a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:17.113115 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-271f5cfd-4135-4dd6-9352-deb629802de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:17.179310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1156/2330] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:17.183845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fafba-63ef-4496-b308-e361713e8c79 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:17.244473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fafba-63ef-4496-b308-e361713e8c79 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:17.249436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbb073-8ff7-4b10-a5c7-9949658f49af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:17.250154 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbb073-8ff7-4b10-a5c7-9949658f49af admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:17.250465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbb073-8ff7-4b10-a5c7-9949658f49af admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:17.250767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbb073-8ff7-4b10-a5c7-9949658f49af admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:17.251480 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbb073-8ff7-4b10-a5c7-9949658f49af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:17.253556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbb073-8ff7-4b10-a5c7-9949658f49af admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:17.271814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:17.281984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1175/2331] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:26:17 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:17.284388 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f5cfd-4135-4dd6-9352-deb629802de3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:17.296685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f5cfd-4135-4dd6-9352-deb629802de3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:17.325512 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:17.354941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f1ffe-709f-4179-968d-bb1f37f37708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:17.355651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f1ffe-709f-4179-968d-bb1f37f37708 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:17.355939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f1ffe-709f-4179-968d-bb1f37f37708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:17.356216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f1ffe-709f-4179-968d-bb1f37f37708 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:17.356920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f1ffe-709f-4179-968d-bb1f37f37708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:17.358980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f1ffe-709f-4179-968d-bb1f37f37708 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:17.379205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1157/2332] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:26:17 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:17.383331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbb073-8ff7-4b10-a5c7-9949658f49af admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:17.394289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbb073-8ff7-4b10-a5c7-9949658f49af admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:17.399176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd1112-2fd5-4b90-b401-4e7cd4d02568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:17.399840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd1112-2fd5-4b90-b401-4e7cd4d02568 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:17.400126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd1112-2fd5-4b90-b401-4e7cd4d02568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:17.400405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd1112-2fd5-4b90-b401-4e7cd4d02568 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:17.401766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd1112-2fd5-4b90-b401-4e7cd4d02568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:17.403542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd1112-2fd5-4b90-b401-4e7cd4d02568 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:17.414732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1176/2333] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:17.419385 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f1ffe-709f-4179-968d-bb1f37f37708 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:17.430485 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f1ffe-709f-4179-968d-bb1f37f37708 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:17.436002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d7eb5-c2cf-4ab0-9746-4498aad2b2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:17.436908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7eb5-c2cf-4ab0-9746-4498aad2b2fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:17.437189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7eb5-c2cf-4ab0-9746-4498aad2b2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:17.437474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7eb5-c2cf-4ab0-9746-4498aad2b2fc admin admin] PATH_INFO: `/v3/projects/9de4a1e4a15642319a5b4475d99f8e16/users/11650b450fcb4ba2b1e5f34a9a0b8d05/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:17.455307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d7eb5-c2cf-4ab0-9746-4498aad2b2fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de4a1e4a15642319a5b4475d99f8e16, user_id=11650b450fcb4ba2b1e5f34a9a0b8d05, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:17.458235 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d7eb5-c2cf-4ab0-9746-4498aad2b2fc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:17.472131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1158/2334] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:17 2022] PUT /identity/v3/projects/9de4a1e4a15642319a5b4475d99f8e16/users/11650b450fcb4ba2b1e5f34a9a0b8d05/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:17.476204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd1112-2fd5-4b90-b401-4e7cd4d02568 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:17.486376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd1112-2fd5-4b90-b401-4e7cd4d02568 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:17.544086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fb4e7-4a96-4f74-8aea-f66df990269d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:17.544751 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fb4e7-4a96-4f74-8aea-f66df990269d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:17.545035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fb4e7-4a96-4f74-8aea-f66df990269d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:17.545314 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fb4e7-4a96-4f74-8aea-f66df990269d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:17.546073 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fb4e7-4a96-4f74-8aea-f66df990269d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:17.547685 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fb4e7-4a96-4f74-8aea-f66df990269d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:17.558078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1177/2335] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:17.562282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d7eb5-c2cf-4ab0-9746-4498aad2b2fc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:17.572509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d7eb5-c2cf-4ab0-9746-4498aad2b2fc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:17.577691 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f7f13-899e-4564-9e8d-1c4abad689e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:17.578601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f7f13-899e-4564-9e8d-1c4abad689e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:17.578886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f7f13-899e-4564-9e8d-1c4abad689e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:17.579162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f7f13-899e-4564-9e8d-1c4abad689e7 admin admin] PATH_INFO: `/v3/projects/9de4a1e4a15642319a5b4475d99f8e16/users/11650b450fcb4ba2b1e5f34a9a0b8d05/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:17.581173 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f7f13-899e-4564-9e8d-1c4abad689e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de4a1e4a15642319a5b4475d99f8e16, user_id=11650b450fcb4ba2b1e5f34a9a0b8d05, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:17.583858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f7f13-899e-4564-9e8d-1c4abad689e7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:17.595352 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1159/2336] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:17 2022] PUT /identity/v3/projects/9de4a1e4a15642319a5b4475d99f8e16/users/11650b450fcb4ba2b1e5f34a9a0b8d05/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:17.606701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3d586-6101-4e58-8ddb-90b7b828923e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:17.606992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3d586-6101-4e58-8ddb-90b7b828923e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:17.607275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3d586-6101-4e58-8ddb-90b7b828923e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:17.633168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d5d3d586-6101-4e58-8ddb-90b7b828923e None None] MFA Rules not processed for user `11650b450fcb4ba2b1e5f34a9a0b8d05`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:17.634461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d5d3d586-6101-4e58-8ddb-90b7b828923e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:17.701976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1178/2337] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:17.714812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f7f13-899e-4564-9e8d-1c4abad689e7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:17.716448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d91f7f13-899e-4564-9e8d-1c4abad689e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:17.772036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f7f13-899e-4564-9e8d-1c4abad689e7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:17.775713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9259ef6-ab28-4811-9bb8-9d9bbb7b169b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:17.776903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9259ef6-ab28-4811-9bb8-9d9bbb7b169b service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:17.777182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9259ef6-ab28-4811-9bb8-9d9bbb7b169b service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:17.777461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9259ef6-ab28-4811-9bb8-9d9bbb7b169b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:17.786814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9259ef6-ab28-4811-9bb8-9d9bbb7b169b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:17.788250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9259ef6-ab28-4811-9bb8-9d9bbb7b169b service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:17.798317 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1160/2338] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:18.114418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3d586-6101-4e58-8ddb-90b7b828923e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:18.124197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3d586-6101-4e58-8ddb-90b7b828923e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:18.128820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525458af-998e-4e0f-80d7-fb452f4e1f97 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11650b450fcb4ba2b1e5f34a9a0b8d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9de4a1e4a15642319a5b4475d99f8e16', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:18.129453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525458af-998e-4e0f-80d7-fb452f4e1f97 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:18.129770 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525458af-998e-4e0f-80d7-fb452f4e1f97 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:18.130053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525458af-998e-4e0f-80d7-fb452f4e1f97 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:18.130782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525458af-998e-4e0f-80d7-fb452f4e1f97 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:26:18.131121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525458af-998e-4e0f-80d7-fb452f4e1f97 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:18.132723 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525458af-998e-4e0f-80d7-fb452f4e1f97 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:18.148004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1179/2339] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:26:18.149188 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9259ef6-ab28-4811-9bb8-9d9bbb7b169b service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:18.159313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9259ef6-ab28-4811-9bb8-9d9bbb7b169b service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:18.164154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06ae9a-c640-4abf-8758-0be01cc6c4da tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11650b450fcb4ba2b1e5f34a9a0b8d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9de4a1e4a15642319a5b4475d99f8e16', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:18.164799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06ae9a-c640-4abf-8758-0be01cc6c4da tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:18.165080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06ae9a-c640-4abf-8758-0be01cc6c4da tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:18.165351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06ae9a-c640-4abf-8758-0be01cc6c4da tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:18.166146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06ae9a-c640-4abf-8758-0be01cc6c4da tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:18.168931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06ae9a-c640-4abf-8758-0be01cc6c4da tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:18.195774 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1161/2340] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:26:18 2022] POST /identity/v3/projects => generated 353 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:26:18.200167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525458af-998e-4e0f-80d7-fb452f4e1f97 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:18.209965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525458af-998e-4e0f-80d7-fb452f4e1f97 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:18.263715 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:18.271651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a772a-6645-4385-85c3-672a05ae3bb3 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11650b450fcb4ba2b1e5f34a9a0b8d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9de4a1e4a15642319a5b4475d99f8e16', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:18.272320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a772a-6645-4385-85c3-672a05ae3bb3 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:18.272604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a772a-6645-4385-85c3-672a05ae3bb3 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:18.272882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a772a-6645-4385-85c3-672a05ae3bb3 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:18.273612 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a772a-6645-4385-85c3-672a05ae3bb3 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:26:18.273953 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a772a-6645-4385-85c3-672a05ae3bb3 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:18.275581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a772a-6645-4385-85c3-672a05ae3bb3 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:18.288017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1180/2341] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:26:18.325551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06ae9a-c640-4abf-8758-0be01cc6c4da tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:18.335607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06ae9a-c640-4abf-8758-0be01cc6c4da tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:18.340394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11650b450fcb4ba2b1e5f34a9a0b8d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9de4a1e4a15642319a5b4475d99f8e16', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:18.341247 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:18.341534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:18.341846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] PATH_INFO: `/v3/projects/a262bafb815e41c7be0cba3feeb1f978` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:18.343028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a262bafb815e41c7be0cba3feeb1f978)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:18.344967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:18.394750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a262bafb815e41c7be0cba3feeb1f978', 'request_id': 'req-6a0e5a75-b022-4946-a139-2ef430d9cef6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:18.400381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a262bafb815e41c7be0cba3feeb1f978', 'request_id': 'req-6a0e5a75-b022-4946-a139-2ef430d9cef6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:18.402554 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:18.407511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a262bafb815e41c7be0cba3feeb1f978', 'request_id': 'req-6a0e5a75-b022-4946-a139-2ef430d9cef6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:18.421670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] The token cache is being invalidate because project a262bafb815e41c7be0cba3feeb1f978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:18.421987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:18.422627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6a0e5a75-b022-4946-a139-2ef430d9cef6 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:18.424180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1162/2342] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:18 2022] DELETE /identity/v3/projects/a262bafb815e41c7be0cba3feeb1f978 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:18.636521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a772a-6645-4385-85c3-672a05ae3bb3 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:18.638411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-203a772a-6645-4385-85c3-672a05ae3bb3 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:18.712950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a772a-6645-4385-85c3-672a05ae3bb3 tempest-VolumeQuotaClassesTest-768332957 tempest-VolumeQuotaClassesTest-768332957-project-admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:18.767974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:18.768888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:18.769169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:18.769442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] PATH_INFO: `/v3/users/1c1d0d65b960496780853910208e8a9d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:18.770646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1d0d65b960496780853910208e8a9d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:18.772563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:18.818656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1d0d65b960496780853910208e8a9d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:18.819312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1d0d65b960496780853910208e8a9d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:18.819926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1d0d65b960496780853910208e8a9d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:18.831418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1d0d65b960496780853910208e8a9d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:18.838105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1d0d65b960496780853910208e8a9d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:18.851436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1181/2343] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:26:18 2022] DELETE /identity/v3/users/1c1d0d65b960496780853910208e8a9d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:18.857033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925be760-f6cd-4c4e-93e3-cc6521f8f878 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:18.857626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925be760-f6cd-4c4e-93e3-cc6521f8f878 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:18.857626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925be760-f6cd-4c4e-93e3-cc6521f8f878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:18.875294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-925be760-f6cd-4c4e-93e3-cc6521f8f878 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:18.876478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-925be760-f6cd-4c4e-93e3-cc6521f8f878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:18.936595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1163/2344] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:18.950231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:18.951964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:18.971061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:19.011495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c501a1-47b4-4678-8b99-51278c913660 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:19.015506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397ae46-d26a-442c-9b12-9ebfc20690c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:19.016191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397ae46-d26a-442c-9b12-9ebfc20690c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:19.016486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397ae46-d26a-442c-9b12-9ebfc20690c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:19.016777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397ae46-d26a-442c-9b12-9ebfc20690c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:19.026125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397ae46-d26a-442c-9b12-9ebfc20690c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:19.028272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397ae46-d26a-442c-9b12-9ebfc20690c7 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:19.038292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1182/2345] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:19.311186 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925be760-f6cd-4c4e-93e3-cc6521f8f878 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:19.312906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-925be760-f6cd-4c4e-93e3-cc6521f8f878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:19.342636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925be760-f6cd-4c4e-93e3-cc6521f8f878 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:19.348118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:19.349116 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:19.349536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:19.350016 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] PATH_INFO: `/v3/projects/834e969de42342039fe817db706f5043` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:19.351535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834e969de42342039fe817db706f5043)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:19.353856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:19.413749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834e969de42342039fe817db706f5043', 'request_id': 'req-aca84220-50c0-48af-8287-f37118473766'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:19.426201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834e969de42342039fe817db706f5043', 'request_id': 'req-aca84220-50c0-48af-8287-f37118473766'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:19.435631 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834e969de42342039fe817db706f5043', 'request_id': 'req-aca84220-50c0-48af-8287-f37118473766'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:19.452379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] The token cache is being invalidate because project 834e969de42342039fe817db706f5043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:19.452908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:19.453806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:19.455677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1164/2346] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:19 2022] DELETE /identity/v3/projects/834e969de42342039fe817db706f5043 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:19.459885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397ae46-d26a-442c-9b12-9ebfc20690c7 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:19.470436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397ae46-d26a-442c-9b12-9ebfc20690c7 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:19.532103 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6015ba-c530-44d4-880e-4ae5f3df3386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:19.532980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6015ba-c530-44d4-880e-4ae5f3df3386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:19.533255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6015ba-c530-44d4-880e-4ae5f3df3386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:19.533526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6015ba-c530-44d4-880e-4ae5f3df3386 admin admin] PATH_INFO: `/v3/users/11650b450fcb4ba2b1e5f34a9a0b8d05` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:19.534737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6015ba-c530-44d4-880e-4ae5f3df3386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11650b450fcb4ba2b1e5f34a9a0b8d05)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:19.536926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6015ba-c530-44d4-880e-4ae5f3df3386 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:19.586836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8c6015ba-c530-44d4-880e-4ae5f3df3386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11650b450fcb4ba2b1e5f34a9a0b8d05'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:19.587486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8c6015ba-c530-44d4-880e-4ae5f3df3386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11650b450fcb4ba2b1e5f34a9a0b8d05'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:19.588086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8c6015ba-c530-44d4-880e-4ae5f3df3386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11650b450fcb4ba2b1e5f34a9a0b8d05'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:19.600598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8c6015ba-c530-44d4-880e-4ae5f3df3386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11650b450fcb4ba2b1e5f34a9a0b8d05'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:19.607727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8c6015ba-c530-44d4-880e-4ae5f3df3386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11650b450fcb4ba2b1e5f34a9a0b8d05'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:19.622153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1183/2347] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:26:19 2022] DELETE /identity/v3/users/11650b450fcb4ba2b1e5f34a9a0b8d05 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:20.195640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:20.197185 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:20.210498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca84220-50c0-48af-8287-f37118473766 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:20.291747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:20.293271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:20.293555 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:20.293982 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] PATH_INFO: `/v3/projects/9de4a1e4a15642319a5b4475d99f8e16` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:20.295210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9de4a1e4a15642319a5b4475d99f8e16)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:20.297140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:20.316795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:20.350305 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9de4a1e4a15642319a5b4475d99f8e16', 'request_id': 'req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:20.357221 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9de4a1e4a15642319a5b4475d99f8e16', 'request_id': 'req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:20.364180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9de4a1e4a15642319a5b4475d99f8e16', 'request_id': 'req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:20.377185 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] The token cache is being invalidate because project 9de4a1e4a15642319a5b4475d99f8e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:20.377474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:20.378114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6bed1a4d-aef5-4755-9064-f6ab78ec4e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:20.379667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1165/2348] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:20 2022] DELETE /identity/v3/projects/9de4a1e4a15642319a5b4475d99f8e16 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:20.384522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0092cf4-1166-40f1-a2d7-9cf32cf46c24 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:20.385066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0092cf4-1166-40f1-a2d7-9cf32cf46c24 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:20.385066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0092cf4-1166-40f1-a2d7-9cf32cf46c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:20.404262 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b0092cf4-1166-40f1-a2d7-9cf32cf46c24 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:20.405479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b0092cf4-1166-40f1-a2d7-9cf32cf46c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:20.441509 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:20.473738 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1184/2349] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:20.482085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6351fe-9969-4fd6-8e9e-fc8b7974e6d3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:20.482372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6351fe-9969-4fd6-8e9e-fc8b7974e6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:20.482651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6351fe-9969-4fd6-8e9e-fc8b7974e6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:20.501897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5c6351fe-9969-4fd6-8e9e-fc8b7974e6d3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:20.503177 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5c6351fe-9969-4fd6-8e9e-fc8b7974e6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:20.512062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1166/2350] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:20.516469 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0092cf4-1166-40f1-a2d7-9cf32cf46c24 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:20.527300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0092cf4-1166-40f1-a2d7-9cf32cf46c24 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:20.532363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c2266-d89d-4498-a11e-b319d04d961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:20.533046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c2266-d89d-4498-a11e-b319d04d961b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:20.533335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c2266-d89d-4498-a11e-b319d04d961b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:20.533620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c2266-d89d-4498-a11e-b319d04d961b admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:20.534403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c2266-d89d-4498-a11e-b319d04d961b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:26:20.534712 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c2266-d89d-4498-a11e-b319d04d961b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:20.536352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c2266-d89d-4498-a11e-b319d04d961b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:20.549769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1185/2351] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:26:20.554048 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6351fe-9969-4fd6-8e9e-fc8b7974e6d3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:20.563985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6351fe-9969-4fd6-8e9e-fc8b7974e6d3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:20.568946 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60130ac1-7430-4e16-856e-a0dda64ee0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:20.569806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60130ac1-7430-4e16-856e-a0dda64ee0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:20.570161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60130ac1-7430-4e16-856e-a0dda64ee0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:20.570595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60130ac1-7430-4e16-856e-a0dda64ee0f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:20.571312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60130ac1-7430-4e16-856e-a0dda64ee0f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:20.573803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60130ac1-7430-4e16-856e-a0dda64ee0f9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:20.606595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1167/2352] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:26:20 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:20.610755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c2266-d89d-4498-a11e-b319d04d961b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:20.621107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c2266-d89d-4498-a11e-b319d04d961b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:20.684846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed242c-e2f8-4620-b0e9-76a169614b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:20.685525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed242c-e2f8-4620-b0e9-76a169614b3f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:20.685844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed242c-e2f8-4620-b0e9-76a169614b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:20.686126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed242c-e2f8-4620-b0e9-76a169614b3f admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:20.686814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed242c-e2f8-4620-b0e9-76a169614b3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:20.688746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed242c-e2f8-4620-b0e9-76a169614b3f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:20.714370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1186/2353] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:26:20 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:20.715581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60130ac1-7430-4e16-856e-a0dda64ee0f9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:20.728548 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60130ac1-7430-4e16-856e-a0dda64ee0f9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:20.733409 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af403a37-6cbc-4ca7-84dd-1d696a48190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:20.734139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af403a37-6cbc-4ca7-84dd-1d696a48190d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:20.734427 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af403a37-6cbc-4ca7-84dd-1d696a48190d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:20.734705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af403a37-6cbc-4ca7-84dd-1d696a48190d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:20.735463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af403a37-6cbc-4ca7-84dd-1d696a48190d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:20.737068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af403a37-6cbc-4ca7-84dd-1d696a48190d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:20.747926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1168/2354] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:20.752227 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed242c-e2f8-4620-b0e9-76a169614b3f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:20.762857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed242c-e2f8-4620-b0e9-76a169614b3f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:20.768661 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cd8e6-e9ce-4f3b-ba3f-560ec6676c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:20.769464 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cd8e6-e9ce-4f3b-ba3f-560ec6676c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:20.770038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cd8e6-e9ce-4f3b-ba3f-560ec6676c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:20.770428 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cd8e6-e9ce-4f3b-ba3f-560ec6676c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:20.771620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cd8e6-e9ce-4f3b-ba3f-560ec6676c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:20.773403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cd8e6-e9ce-4f3b-ba3f-560ec6676c9d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:20.785371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1187/2355] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:26:20.790103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af403a37-6cbc-4ca7-84dd-1d696a48190d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:20.800957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af403a37-6cbc-4ca7-84dd-1d696a48190d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:20.806391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0a9f3-91cc-4791-aae6-dcb955729cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:20.807798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0a9f3-91cc-4791-aae6-dcb955729cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:20.808103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0a9f3-91cc-4791-aae6-dcb955729cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:20.808361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0a9f3-91cc-4791-aae6-dcb955729cc3 admin admin] PATH_INFO: `/v3/projects/17fc056baaff4111a34192dd521463ed/users/2ea60a99bd164a69a3033fa17081fd60/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:20.823623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0a9f3-91cc-4791-aae6-dcb955729cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fc056baaff4111a34192dd521463ed, user_id=2ea60a99bd164a69a3033fa17081fd60, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:20.826358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0a9f3-91cc-4791-aae6-dcb955729cc3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:20.839252 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1169/2356] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:20 2022] PUT /identity/v3/projects/17fc056baaff4111a34192dd521463ed/users/2ea60a99bd164a69a3033fa17081fd60/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:20.849872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ab21a-4834-47ca-b490-a10efc0ab35a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:20.850172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ab21a-4834-47ca-b490-a10efc0ab35a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:20.850457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ab21a-4834-47ca-b490-a10efc0ab35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:20.877800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-de0ab21a-4834-47ca-b490-a10efc0ab35a None None] MFA Rules not processed for user `2ea60a99bd164a69a3033fa17081fd60`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:20.879020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-de0ab21a-4834-47ca-b490-a10efc0ab35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:20.945550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1188/2357] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:20.951063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0a9f3-91cc-4791-aae6-dcb955729cc3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:21.057670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0a9f3-91cc-4791-aae6-dcb955729cc3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:21.066432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0c06c-47ec-4c96-8982-ac6a33a65526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:21.067199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0c06c-47ec-4c96-8982-ac6a33a65526 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:21.067562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0c06c-47ec-4c96-8982-ac6a33a65526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:21.067909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0c06c-47ec-4c96-8982-ac6a33a65526 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:21.068692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0c06c-47ec-4c96-8982-ac6a33a65526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:21.070910 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0c06c-47ec-4c96-8982-ac6a33a65526 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:21.108666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1170/2358] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:26:20 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:21.112545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ab21a-4834-47ca-b490-a10efc0ab35a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:21.124764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ab21a-4834-47ca-b490-a10efc0ab35a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:21.169581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:21.206697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8f609-3d08-4925-99c7-507598fe388e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:21.207525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8f609-3d08-4925-99c7-507598fe388e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:21.207911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8f609-3d08-4925-99c7-507598fe388e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:21.208260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8f609-3d08-4925-99c7-507598fe388e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:21.209086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8f609-3d08-4925-99c7-507598fe388e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:21.211163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8f609-3d08-4925-99c7-507598fe388e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:21.232998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1189/2359] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:26:21 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:21.237049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0c06c-47ec-4c96-8982-ac6a33a65526 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:21.247936 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0c06c-47ec-4c96-8982-ac6a33a65526 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:21.252627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013f3c2-cdf5-40cf-a086-dc4c7e278913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:21.253297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013f3c2-cdf5-40cf-a086-dc4c7e278913 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:21.253595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013f3c2-cdf5-40cf-a086-dc4c7e278913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:21.253891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013f3c2-cdf5-40cf-a086-dc4c7e278913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:21.254619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013f3c2-cdf5-40cf-a086-dc4c7e278913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:21.256612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013f3c2-cdf5-40cf-a086-dc4c7e278913 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:21.271079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1171/2360] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:26:21.277607 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8f609-3d08-4925-99c7-507598fe388e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:21.289965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8f609-3d08-4925-99c7-507598fe388e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:21.295495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eef57-3a96-4487-817b-732cb2c8add8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:21.296654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eef57-3a96-4487-817b-732cb2c8add8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:21.296930 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eef57-3a96-4487-817b-732cb2c8add8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:21.297204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eef57-3a96-4487-817b-732cb2c8add8 admin admin] PATH_INFO: `/v3/projects/c3b95d58a9b44a2f9582594dba41ba9f/users/fc9dfa380c1b4b15b71d66b827ac26f2/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:21.313951 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3eef57-3a96-4487-817b-732cb2c8add8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b95d58a9b44a2f9582594dba41ba9f, user_id=fc9dfa380c1b4b15b71d66b827ac26f2, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:21.316645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3eef57-3a96-4487-817b-732cb2c8add8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:21.331591 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1190/2361] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:21 2022] PUT /identity/v3/projects/c3b95d58a9b44a2f9582594dba41ba9f/users/fc9dfa380c1b4b15b71d66b827ac26f2/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:21.335644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013f3c2-cdf5-40cf-a086-dc4c7e278913 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:21.347358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013f3c2-cdf5-40cf-a086-dc4c7e278913 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:21.357424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:21.438345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3b5d6-e8b8-49a2-8374-55074d8d83c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:21.439127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3b5d6-e8b8-49a2-8374-55074d8d83c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:21.439484 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3b5d6-e8b8-49a2-8374-55074d8d83c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:21.439843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3b5d6-e8b8-49a2-8374-55074d8d83c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:21.441492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3b5d6-e8b8-49a2-8374-55074d8d83c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:21.442581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3b5d6-e8b8-49a2-8374-55074d8d83c4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:21.459739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1172/2362] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:21.463496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eef57-3a96-4487-817b-732cb2c8add8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:21.479928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eef57-3a96-4487-817b-732cb2c8add8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:21.486410 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82aba84-71b8-4d40-9389-707d10102311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:21.487468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aba84-71b8-4d40-9389-707d10102311 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:21.487754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aba84-71b8-4d40-9389-707d10102311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:21.488032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aba84-71b8-4d40-9389-707d10102311 admin admin] PATH_INFO: `/v3/projects/c3b95d58a9b44a2f9582594dba41ba9f/users/fc9dfa380c1b4b15b71d66b827ac26f2/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:21.490420 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82aba84-71b8-4d40-9389-707d10102311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b95d58a9b44a2f9582594dba41ba9f, user_id=fc9dfa380c1b4b15b71d66b827ac26f2, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:21.493350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82aba84-71b8-4d40-9389-707d10102311 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:21.517957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1191/2363] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:21 2022] PUT /identity/v3/projects/c3b95d58a9b44a2f9582594dba41ba9f/users/fc9dfa380c1b4b15b71d66b827ac26f2/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:26:21.519922 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc76ac-1bc4-4eee-a40c-ed9a4f91751b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:21.520815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc76ac-1bc4-4eee-a40c-ed9a4f91751b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:21.521202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc76ac-1bc4-4eee-a40c-ed9a4f91751b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:21.567282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-eddc76ac-1bc4-4eee-a40c-ed9a4f91751b None None] MFA Rules not processed for user `fc9dfa380c1b4b15b71d66b827ac26f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:21.569065 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-eddc76ac-1bc4-4eee-a40c-ed9a4f91751b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:21.653759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1173/2364] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:21.665409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82aba84-71b8-4d40-9389-707d10102311 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:21.667216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b82aba84-71b8-4d40-9389-707d10102311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:21.732561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82aba84-71b8-4d40-9389-707d10102311 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:21.736188 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab52d6-8e89-4f50-bdbd-b9ac99fc9ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:21.736833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab52d6-8e89-4f50-bdbd-b9ac99fc9ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:21.737697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab52d6-8e89-4f50-bdbd-b9ac99fc9ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:21.737697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab52d6-8e89-4f50-bdbd-b9ac99fc9ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:21.747637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab52d6-8e89-4f50-bdbd-b9ac99fc9ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:21.749090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab52d6-8e89-4f50-bdbd-b9ac99fc9ef8 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:21.759347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1192/2365] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:22.149414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc76ac-1bc4-4eee-a40c-ed9a4f91751b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:22.231398 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc76ac-1bc4-4eee-a40c-ed9a4f91751b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:22.237191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:22.240742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:22.241114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:22.241478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] PATH_INFO: `/v3/users/2ea60a99bd164a69a3033fa17081fd60` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:22.243110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea60a99bd164a69a3033fa17081fd60)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:22.245345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:22.295403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:22.305688 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea60a99bd164a69a3033fa17081fd60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:22.314839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea60a99bd164a69a3033fa17081fd60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:22.315740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea60a99bd164a69a3033fa17081fd60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:22.329368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea60a99bd164a69a3033fa17081fd60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:22.336359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea60a99bd164a69a3033fa17081fd60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:22.342867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1174/2366] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:26:22 2022] DELETE /identity/v3/users/2ea60a99bd164a69a3033fa17081fd60 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:22.348220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9aa7f-e958-4da5-ba33-caef60697ead None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:22.348596 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9aa7f-e958-4da5-ba33-caef60697ead None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:22.348961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9aa7f-e958-4da5-ba33-caef60697ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:22.371634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d4e9aa7f-e958-4da5-ba33-caef60697ead None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:22.373012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d4e9aa7f-e958-4da5-ba33-caef60697ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:22.441540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1193/2367] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:22.452016 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:22.453458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:22.509322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c152d-cb8c-4c90-acc1-1b5265ab1372 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:22.513495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662b154-bf3f-43ef-8c7c-d1522fadecab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:22.514187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b154-bf3f-43ef-8c7c-d1522fadecab service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:22.514466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b154-bf3f-43ef-8c7c-d1522fadecab service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:22.514738 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b154-bf3f-43ef-8c7c-d1522fadecab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:22.523580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662b154-bf3f-43ef-8c7c-d1522fadecab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:22.525011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662b154-bf3f-43ef-8c7c-d1522fadecab service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:22.534573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1175/2368] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:22.819809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9aa7f-e958-4da5-ba33-caef60697ead None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:22.821317 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d4e9aa7f-e958-4da5-ba33-caef60697ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:22.833657 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9aa7f-e958-4da5-ba33-caef60697ead None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:22.839808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:22.840686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:22.840969 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:22.841247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] PATH_INFO: `/v3/projects/17fc056baaff4111a34192dd521463ed` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:22.842454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17fc056baaff4111a34192dd521463ed)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:22.844858 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:22.873237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:22.896598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17fc056baaff4111a34192dd521463ed', 'request_id': 'req-7bf49625-600b-4223-b72e-8020ae951e05'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:22.903025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17fc056baaff4111a34192dd521463ed', 'request_id': 'req-7bf49625-600b-4223-b72e-8020ae951e05'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:22.910503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17fc056baaff4111a34192dd521463ed', 'request_id': 'req-7bf49625-600b-4223-b72e-8020ae951e05'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:22.921838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] The token cache is being invalidate because project 17fc056baaff4111a34192dd521463ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:22.922123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:22.922746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:22.924271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1194/2369] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:22 2022] DELETE /identity/v3/projects/17fc056baaff4111a34192dd521463ed => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:22.928469 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662b154-bf3f-43ef-8c7c-d1522fadecab service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:22.939737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662b154-bf3f-43ef-8c7c-d1522fadecab service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:23.006275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a40f2-1936-4186-8e16-413620bb2dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:23.007193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a40f2-1936-4186-8e16-413620bb2dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:23.007479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a40f2-1936-4186-8e16-413620bb2dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:23.007760 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a40f2-1936-4186-8e16-413620bb2dbb admin admin] PATH_INFO: `/v3/users/fc9dfa380c1b4b15b71d66b827ac26f2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:23.008952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a40f2-1936-4186-8e16-413620bb2dbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc9dfa380c1b4b15b71d66b827ac26f2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:23.010967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a40f2-1936-4186-8e16-413620bb2dbb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:23.058161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d62a40f2-1936-4186-8e16-413620bb2dbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc9dfa380c1b4b15b71d66b827ac26f2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:23.065329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d62a40f2-1936-4186-8e16-413620bb2dbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc9dfa380c1b4b15b71d66b827ac26f2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:23.066009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d62a40f2-1936-4186-8e16-413620bb2dbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc9dfa380c1b4b15b71d66b827ac26f2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:23.070752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:23.081260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d62a40f2-1936-4186-8e16-413620bb2dbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc9dfa380c1b4b15b71d66b827ac26f2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:23.088069 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d62a40f2-1936-4186-8e16-413620bb2dbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc9dfa380c1b4b15b71d66b827ac26f2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:23.095061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1176/2370] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:26:22 2022] DELETE /identity/v3/users/fc9dfa380c1b4b15b71d66b827ac26f2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:23.365473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:23.366906 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:23.382391 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf49625-600b-4223-b72e-8020ae951e05 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:23.445757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:23.446621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:23.446899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:23.447171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] PATH_INFO: `/v3/projects/c3b95d58a9b44a2f9582594dba41ba9f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:23.448861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3b95d58a9b44a2f9582594dba41ba9f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:23.450899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:23.498097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3b95d58a9b44a2f9582594dba41ba9f', 'request_id': 'req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:23.504090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3b95d58a9b44a2f9582594dba41ba9f', 'request_id': 'req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:23.512347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3b95d58a9b44a2f9582594dba41ba9f', 'request_id': 'req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:23.525177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] The token cache is being invalidate because project c3b95d58a9b44a2f9582594dba41ba9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:23.525467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:23.527024 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f5cedaf2-47c2-4f2d-b951-9e6df3f510d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:23.528719 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1195/2371] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:23 2022] DELETE /identity/v3/projects/c3b95d58a9b44a2f9582594dba41ba9f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:23.533661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389a344-c3d4-45fb-9c40-f84a5ddde1cd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:23.534139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389a344-c3d4-45fb-9c40-f84a5ddde1cd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:23.534374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389a344-c3d4-45fb-9c40-f84a5ddde1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:23.552185 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6389a344-c3d4-45fb-9c40-f84a5ddde1cd None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:23.553368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6389a344-c3d4-45fb-9c40-f84a5ddde1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:23.613150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1177/2372] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:23.621239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3ec1-d0d7-4c30-b0ed-6098346e0587 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:23.621552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3ec1-d0d7-4c30-b0ed-6098346e0587 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:23.621920 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3ec1-d0d7-4c30-b0ed-6098346e0587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:23.641403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-013c3ec1-d0d7-4c30-b0ed-6098346e0587 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:23.642664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-013c3ec1-d0d7-4c30-b0ed-6098346e0587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:23.650671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1196/2373] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:23.655828 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389a344-c3d4-45fb-9c40-f84a5ddde1cd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:23.665635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389a344-c3d4-45fb-9c40-f84a5ddde1cd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:23.670607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9fdee-1944-4c74-818d-6e2e58dfcf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:23.671304 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9fdee-1944-4c74-818d-6e2e58dfcf67 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:23.671590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9fdee-1944-4c74-818d-6e2e58dfcf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:23.671877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9fdee-1944-4c74-818d-6e2e58dfcf67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:23.672664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9fdee-1944-4c74-818d-6e2e58dfcf67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:26:23.672973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9fdee-1944-4c74-818d-6e2e58dfcf67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:23.674635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9fdee-1944-4c74-818d-6e2e58dfcf67 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:23.686314 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1178/2374] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:26:23.690553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c3ec1-d0d7-4c30-b0ed-6098346e0587 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:23.700085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c3ec1-d0d7-4c30-b0ed-6098346e0587 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:23.704850 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9f245-9c5c-41ba-bbd9-51b8ffcad8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:23.705500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9f245-9c5c-41ba-bbd9-51b8ffcad8c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:23.705826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9f245-9c5c-41ba-bbd9-51b8ffcad8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:23.706107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9f245-9c5c-41ba-bbd9-51b8ffcad8c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:23.706810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9f245-9c5c-41ba-bbd9-51b8ffcad8c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:23.708720 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9f245-9c5c-41ba-bbd9-51b8ffcad8c6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:23.734970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1197/2375] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:26:23 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:23.739514 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9fdee-1944-4c74-818d-6e2e58dfcf67 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:23.750812 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9fdee-1944-4c74-818d-6e2e58dfcf67 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:23.821332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e812eb-f2a1-4f33-88d1-9c5b5302399f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:23.822037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e812eb-f2a1-4f33-88d1-9c5b5302399f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:23.822318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e812eb-f2a1-4f33-88d1-9c5b5302399f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:23.822590 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e812eb-f2a1-4f33-88d1-9c5b5302399f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:23.824950 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e812eb-f2a1-4f33-88d1-9c5b5302399f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:23.827346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e812eb-f2a1-4f33-88d1-9c5b5302399f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:23.847009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1179/2376] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:26:23 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:23.851263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9f245-9c5c-41ba-bbd9-51b8ffcad8c6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:23.861653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9f245-9c5c-41ba-bbd9-51b8ffcad8c6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:23.866871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca948b8-89e4-4e54-9cdf-6441f5ad2b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:23.867467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca948b8-89e4-4e54-9cdf-6441f5ad2b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:23.867748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca948b8-89e4-4e54-9cdf-6441f5ad2b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:23.868021 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca948b8-89e4-4e54-9cdf-6441f5ad2b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:23.868742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca948b8-89e4-4e54-9cdf-6441f5ad2b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:23.871369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca948b8-89e4-4e54-9cdf-6441f5ad2b7d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:23.881536 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1198/2377] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:23.886483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e812eb-f2a1-4f33-88d1-9c5b5302399f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:23.896751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e812eb-f2a1-4f33-88d1-9c5b5302399f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:23.901626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc4ea2-3263-4848-8f10-b9e756edaa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:23.902322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc4ea2-3263-4848-8f10-b9e756edaa09 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:23.902606 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc4ea2-3263-4848-8f10-b9e756edaa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:23.902900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc4ea2-3263-4848-8f10-b9e756edaa09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:23.903666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc4ea2-3263-4848-8f10-b9e756edaa09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:23.905346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc4ea2-3263-4848-8f10-b9e756edaa09 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:23.916284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1180/2378] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:23.920294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca948b8-89e4-4e54-9cdf-6441f5ad2b7d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:23.925399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:23.932718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca948b8-89e4-4e54-9cdf-6441f5ad2b7d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:23.938826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70201b6-97e9-453c-8690-a1c2b742f966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:23.939839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70201b6-97e9-453c-8690-a1c2b742f966 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:23.940186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70201b6-97e9-453c-8690-a1c2b742f966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:23.940532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70201b6-97e9-453c-8690-a1c2b742f966 admin admin] PATH_INFO: `/v3/projects/840b1cebd5ab45f8b4cbd2ebf0dc3bbd/users/8b230eb22362473f9895349220f4744e/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:23.957478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70201b6-97e9-453c-8690-a1c2b742f966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=840b1cebd5ab45f8b4cbd2ebf0dc3bbd, user_id=8b230eb22362473f9895349220f4744e, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:23.960726 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70201b6-97e9-453c-8690-a1c2b742f966 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:23.974680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1199/2379] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:23 2022] PUT /identity/v3/projects/840b1cebd5ab45f8b4cbd2ebf0dc3bbd/users/8b230eb22362473f9895349220f4744e/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:23.986433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3b4a1-e9f3-49a6-8fe1-0f3765897852 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:23.986762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3b4a1-e9f3-49a6-8fe1-0f3765897852 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:23.987045 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3b4a1-e9f3-49a6-8fe1-0f3765897852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:24.015526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7ea3b4a1-e9f3-49a6-8fe1-0f3765897852 None None] MFA Rules not processed for user `8b230eb22362473f9895349220f4744e`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:24.016781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7ea3b4a1-e9f3-49a6-8fe1-0f3765897852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:24.048837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:24.088713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1181/2380] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:24.093483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70201b6-97e9-453c-8690-a1c2b742f966 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:24.157488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70201b6-97e9-453c-8690-a1c2b742f966 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:24.162431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5d63c-0726-40e6-9e7a-e7e01b076d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:24.163091 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5d63c-0726-40e6-9e7a-e7e01b076d5b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:24.163377 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5d63c-0726-40e6-9e7a-e7e01b076d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:24.163654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5d63c-0726-40e6-9e7a-e7e01b076d5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:24.164353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5d63c-0726-40e6-9e7a-e7e01b076d5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:24.166313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5d63c-0726-40e6-9e7a-e7e01b076d5b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:24.197020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1200/2381] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:26:24 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:26:24.198231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3b4a1-e9f3-49a6-8fe1-0f3765897852 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:24.208806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3b4a1-e9f3-49a6-8fe1-0f3765897852 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:24.269240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bfe5d-b5a9-4921-b0b4-3ac78e030850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:24.270294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bfe5d-b5a9-4921-b0b4-3ac78e030850 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:24.270669 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bfe5d-b5a9-4921-b0b4-3ac78e030850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:24.271067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bfe5d-b5a9-4921-b0b4-3ac78e030850 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:24.271837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bfe5d-b5a9-4921-b0b4-3ac78e030850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:24.273929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bfe5d-b5a9-4921-b0b4-3ac78e030850 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:24.295834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1182/2382] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:26:24 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:26:24.299444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5d63c-0726-40e6-9e7a-e7e01b076d5b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:24.311590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5d63c-0726-40e6-9e7a-e7e01b076d5b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:24.317707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546720a1-346f-4707-9ce4-d948e40fd5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:24.318501 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546720a1-346f-4707-9ce4-d948e40fd5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:24.318852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546720a1-346f-4707-9ce4-d948e40fd5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:24.319226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546720a1-346f-4707-9ce4-d948e40fd5c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:24.320031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546720a1-346f-4707-9ce4-d948e40fd5c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:24.321750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546720a1-346f-4707-9ce4-d948e40fd5c2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:24.332824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1201/2383] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:24.336964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bfe5d-b5a9-4921-b0b4-3ac78e030850 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:24.347365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bfe5d-b5a9-4921-b0b4-3ac78e030850 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:24.352032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf7802-e889-441c-b579-c35f4da86275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:24.352908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf7802-e889-441c-b579-c35f4da86275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:24.353200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf7802-e889-441c-b579-c35f4da86275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:24.353479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf7802-e889-441c-b579-c35f4da86275 admin admin] PATH_INFO: `/v3/projects/64b2f5e0a59a4c79bac16b49bc7ca3d8/users/854d7f4b0c2c479abcf801779ac1ef51/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:24.368823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf7802-e889-441c-b579-c35f4da86275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b2f5e0a59a4c79bac16b49bc7ca3d8, user_id=854d7f4b0c2c479abcf801779ac1ef51, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:24.372369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf7802-e889-441c-b579-c35f4da86275 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:24.386856 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1183/2384] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:24 2022] PUT /identity/v3/projects/64b2f5e0a59a4c79bac16b49bc7ca3d8/users/854d7f4b0c2c479abcf801779ac1ef51/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:24.391242 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546720a1-346f-4707-9ce4-d948e40fd5c2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:24.402535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546720a1-346f-4707-9ce4-d948e40fd5c2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:24.461547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e587e3-0084-4eb3-a8bb-80cc42bf81dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:24.462239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e587e3-0084-4eb3-a8bb-80cc42bf81dd admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:24.462521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e587e3-0084-4eb3-a8bb-80cc42bf81dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:24.462794 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e587e3-0084-4eb3-a8bb-80cc42bf81dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:24.463747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e587e3-0084-4eb3-a8bb-80cc42bf81dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:24.465662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e587e3-0084-4eb3-a8bb-80cc42bf81dd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:24.475808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1202/2385] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:24.480686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf7802-e889-441c-b579-c35f4da86275 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:24.491712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf7802-e889-441c-b579-c35f4da86275 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:24.497843 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c059be-1455-4f1c-b845-60d90f1f915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:24.498757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c059be-1455-4f1c-b845-60d90f1f915a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:24.499038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c059be-1455-4f1c-b845-60d90f1f915a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:24.499318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c059be-1455-4f1c-b845-60d90f1f915a admin admin] PATH_INFO: `/v3/projects/64b2f5e0a59a4c79bac16b49bc7ca3d8/users/854d7f4b0c2c479abcf801779ac1ef51/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:24.501496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c059be-1455-4f1c-b845-60d90f1f915a admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b2f5e0a59a4c79bac16b49bc7ca3d8, user_id=854d7f4b0c2c479abcf801779ac1ef51, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:24.504899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c059be-1455-4f1c-b845-60d90f1f915a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:24.517093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1184/2386] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:24 2022] PUT /identity/v3/projects/64b2f5e0a59a4c79bac16b49bc7ca3d8/users/854d7f4b0c2c479abcf801779ac1ef51/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:26:24.528422 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca3a56-6bbd-4e17-ae3e-e7d397e42f6c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:24.528714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca3a56-6bbd-4e17-ae3e-e7d397e42f6c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:24.528992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca3a56-6bbd-4e17-ae3e-e7d397e42f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:24.559276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3dca3a56-6bbd-4e17-ae3e-e7d397e42f6c None None] MFA Rules not processed for user `854d7f4b0c2c479abcf801779ac1ef51`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:24.560571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3dca3a56-6bbd-4e17-ae3e-e7d397e42f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:24.603259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:24.646148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1203/2387] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:24.659119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c059be-1455-4f1c-b845-60d90f1f915a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:24.661197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-03c059be-1455-4f1c-b845-60d90f1f915a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:24.720827 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c059be-1455-4f1c-b845-60d90f1f915a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:24.724689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ce925-914b-474e-abbe-c6dbeb6ed9b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:24.725365 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ce925-914b-474e-abbe-c6dbeb6ed9b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:24.725651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ce925-914b-474e-abbe-c6dbeb6ed9b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:24.725958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ce925-914b-474e-abbe-c6dbeb6ed9b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:24.737038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ce925-914b-474e-abbe-c6dbeb6ed9b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:24.738652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ce925-914b-474e-abbe-c6dbeb6ed9b6 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:24.749005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1185/2388] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:24.823023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca3a56-6bbd-4e17-ae3e-e7d397e42f6c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:24.833120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca3a56-6bbd-4e17-ae3e-e7d397e42f6c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:24.836899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c3c04-852d-4420-b9c1-ec6dc179cb86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:24.837541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c3c04-852d-4420-b9c1-ec6dc179cb86 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:24.837854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c3c04-852d-4420-b9c1-ec6dc179cb86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:24.838131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c3c04-852d-4420-b9c1-ec6dc179cb86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:24.904269 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c3c04-852d-4420-b9c1-ec6dc179cb86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:24.905640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c3c04-852d-4420-b9c1-ec6dc179cb86 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:24.915419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1204/2389] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:25.198834 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcee5f-cce1-4223-8c38-dac574549845 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:25.199247 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcee5f-cce1-4223-8c38-dac574549845 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:25.199584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcee5f-cce1-4223-8c38-dac574549845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:25.200456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c3c04-852d-4420-b9c1-ec6dc179cb86 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:25.219536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4fbcee5f-cce1-4223-8c38-dac574549845 None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:25.221307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4fbcee5f-cce1-4223-8c38-dac574549845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:25.252463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:25.276601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c3c04-852d-4420-b9c1-ec6dc179cb86 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:25.283646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d58d93-142e-4833-9379-008e6a7b4842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:25.284975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d58d93-142e-4833-9379-008e6a7b4842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:25.285386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d58d93-142e-4833-9379-008e6a7b4842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:25.285843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d58d93-142e-4833-9379-008e6a7b4842 admin admin] PATH_INFO: `/v3/users/8b230eb22362473f9895349220f4744e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:25.287874 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d58d93-142e-4833-9379-008e6a7b4842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b230eb22362473f9895349220f4744e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:25.290978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d58d93-142e-4833-9379-008e6a7b4842 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:25.294496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1186/2390] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:26:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:25.349206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69d58d93-142e-4833-9379-008e6a7b4842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b230eb22362473f9895349220f4744e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:25.349915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69d58d93-142e-4833-9379-008e6a7b4842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b230eb22362473f9895349220f4744e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:25.350554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69d58d93-142e-4833-9379-008e6a7b4842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b230eb22362473f9895349220f4744e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:25.357619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcee5f-cce1-4223-8c38-dac574549845 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:25.359109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4fbcee5f-cce1-4223-8c38-dac574549845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:25.363637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69d58d93-142e-4833-9379-008e6a7b4842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b230eb22362473f9895349220f4744e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:25.371580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69d58d93-142e-4833-9379-008e6a7b4842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b230eb22362473f9895349220f4744e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:25.373235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:25.387763 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1205/2391] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:26:25 2022] DELETE /identity/v3/users/8b230eb22362473f9895349220f4744e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:25.392112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2245d0e-b48c-49b9-844b-741d5e11c5c8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:25.392410 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2245d0e-b48c-49b9-844b-741d5e11c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:25.392693 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2245d0e-b48c-49b9-844b-741d5e11c5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:25.410533 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a2245d0e-b48c-49b9-844b-741d5e11c5c8 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:25.411977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a2245d0e-b48c-49b9-844b-741d5e11c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:25.440051 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcee5f-cce1-4223-8c38-dac574549845 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:25.483160 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1206/2392] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:26:25.492826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2245d0e-b48c-49b9-844b-741d5e11c5c8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:25.494179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a2245d0e-b48c-49b9-844b-741d5e11c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:25.506140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0ed52-4091-4117-8585-462f81cbe2db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:25.506900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0ed52-4091-4117-8585-462f81cbe2db service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:25.507232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0ed52-4091-4117-8585-462f81cbe2db service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:25.507569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0ed52-4091-4117-8585-462f81cbe2db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:25.509657 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-25d0ed52-4091-4117-8585-462f81cbe2db service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:25.552275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2245d0e-b48c-49b9-844b-741d5e11c5c8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:25.555936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d007bc-77f7-40e4-8693-bf259881edc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:25.556581 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d007bc-77f7-40e4-8693-bf259881edc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:25.556861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d007bc-77f7-40e4-8693-bf259881edc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:25.557687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d007bc-77f7-40e4-8693-bf259881edc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:25.567416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d007bc-77f7-40e4-8693-bf259881edc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:25.568876 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d007bc-77f7-40e4-8693-bf259881edc3 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:25.573653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0ed52-4091-4117-8585-462f81cbe2db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:25.575274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0ed52-4091-4117-8585-462f81cbe2db service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:25.579342 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1207/2393] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:26:25.583386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1187/2394] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:26:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:26:25.986764 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d007bc-77f7-40e4-8693-bf259881edc3 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:25.988756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-71d007bc-77f7-40e4-8693-bf259881edc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:26.018559 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d007bc-77f7-40e4-8693-bf259881edc3 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:26.023227 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:26.024813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:26.024961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:26.025243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] PATH_INFO: `/v3/projects/840b1cebd5ab45f8b4cbd2ebf0dc3bbd` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:26.026996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=840b1cebd5ab45f8b4cbd2ebf0dc3bbd)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:26.029247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:26.083647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840b1cebd5ab45f8b4cbd2ebf0dc3bbd', 'request_id': 'req-6883063a-b076-4c91-b447-150f2bea7679'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.091472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840b1cebd5ab45f8b4cbd2ebf0dc3bbd', 'request_id': 'req-6883063a-b076-4c91-b447-150f2bea7679'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.102293 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840b1cebd5ab45f8b4cbd2ebf0dc3bbd', 'request_id': 'req-6883063a-b076-4c91-b447-150f2bea7679'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.116430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] The token cache is being invalidate because project 840b1cebd5ab45f8b4cbd2ebf0dc3bbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:26.116737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.117387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.122063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1208/2395] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:25 2022] DELETE /identity/v3/projects/840b1cebd5ab45f8b4cbd2ebf0dc3bbd => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:26.124584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0ed52-4091-4117-8585-462f81cbe2db service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:26.143478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0ed52-4091-4117-8585-462f81cbe2db service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:26.168332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:26.227743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ee382-85e1-4c2d-9fef-298cd07ee5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:26.229529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ee382-85e1-4c2d-9fef-298cd07ee5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:26.230361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ee382-85e1-4c2d-9fef-298cd07ee5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:26.230654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ee382-85e1-4c2d-9fef-298cd07ee5df admin admin] PATH_INFO: `/v3/users/854d7f4b0c2c479abcf801779ac1ef51` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:26.232572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ee382-85e1-4c2d-9fef-298cd07ee5df admin admin] RBAC: Authorizing `identity:delete_user(user_id=854d7f4b0c2c479abcf801779ac1ef51)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:26.234737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ee382-85e1-4c2d-9fef-298cd07ee5df admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:26.315882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a73ee382-85e1-4c2d-9fef-298cd07ee5df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854d7f4b0c2c479abcf801779ac1ef51'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.327722 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a73ee382-85e1-4c2d-9fef-298cd07ee5df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854d7f4b0c2c479abcf801779ac1ef51'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.328879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a73ee382-85e1-4c2d-9fef-298cd07ee5df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854d7f4b0c2c479abcf801779ac1ef51'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.348094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a73ee382-85e1-4c2d-9fef-298cd07ee5df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854d7f4b0c2c479abcf801779ac1ef51'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.360626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a73ee382-85e1-4c2d-9fef-298cd07ee5df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854d7f4b0c2c479abcf801779ac1ef51'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.368130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1188/2396] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:26:26 2022] DELETE /identity/v3/users/854d7f4b0c2c479abcf801779ac1ef51 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:26.682683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:26.684516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:26.701665 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883063a-b076-4c91-b447-150f2bea7679 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:26.722411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:26.792306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:26.793206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:26.793487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:26.793788 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] PATH_INFO: `/v3/projects/64b2f5e0a59a4c79bac16b49bc7ca3d8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:26.795134 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] RBAC: Authorizing `identity:delete_project(project_id=64b2f5e0a59a4c79bac16b49bc7ca3d8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:26.797053 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:26.856029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64b2f5e0a59a4c79bac16b49bc7ca3d8', 'request_id': 'req-69846b1d-babe-4a91-8957-37c9d0fdb41f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.863062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64b2f5e0a59a4c79bac16b49bc7ca3d8', 'request_id': 'req-69846b1d-babe-4a91-8957-37c9d0fdb41f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.873978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64b2f5e0a59a4c79bac16b49bc7ca3d8', 'request_id': 'req-69846b1d-babe-4a91-8957-37c9d0fdb41f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.886683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] The token cache is being invalidate because project 64b2f5e0a59a4c79bac16b49bc7ca3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:26.887025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.887613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69846b1d-babe-4a91-8957-37c9d0fdb41f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:26.889216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1209/2397] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:26 2022] DELETE /identity/v3/projects/64b2f5e0a59a4c79bac16b49bc7ca3d8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:26.894445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5012e4-8396-4248-974a-c1b22095fc72 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:26.894808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5012e4-8396-4248-974a-c1b22095fc72 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:26.895217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5012e4-8396-4248-974a-c1b22095fc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:26.915836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0e5012e4-8396-4248-974a-c1b22095fc72 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:26.917215 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0e5012e4-8396-4248-974a-c1b22095fc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:26.998597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1189/2398] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:27.006658 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82b00a-dcc2-40d5-bfab-289e0aad3ead None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:27.007180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82b00a-dcc2-40d5-bfab-289e0aad3ead None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:27.007180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82b00a-dcc2-40d5-bfab-289e0aad3ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:27.035157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bf82b00a-dcc2-40d5-bfab-289e0aad3ead None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:27.037432 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bf82b00a-dcc2-40d5-bfab-289e0aad3ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:27.046908 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1210/2399] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:27.051327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5012e4-8396-4248-974a-c1b22095fc72 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:27.063908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5012e4-8396-4248-974a-c1b22095fc72 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:27.069434 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08f3f5-0d2d-4949-b32c-2cbadf4086a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:27.070258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08f3f5-0d2d-4949-b32c-2cbadf4086a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:27.070647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08f3f5-0d2d-4949-b32c-2cbadf4086a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:27.071026 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08f3f5-0d2d-4949-b32c-2cbadf4086a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:27.071867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08f3f5-0d2d-4949-b32c-2cbadf4086a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:26:27.072277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08f3f5-0d2d-4949-b32c-2cbadf4086a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:27.074747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08f3f5-0d2d-4949-b32c-2cbadf4086a6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:27.102537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82b00a-dcc2-40d5-bfab-289e0aad3ead None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:27.108629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1190/2400] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:26:27.113744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82b00a-dcc2-40d5-bfab-289e0aad3ead None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:27.117491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a4493-12e3-429e-981a-3d1479aa4792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:27.118357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a4493-12e3-429e-981a-3d1479aa4792 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:27.118741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a4493-12e3-429e-981a-3d1479aa4792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:27.119113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a4493-12e3-429e-981a-3d1479aa4792 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:27.120458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a4493-12e3-429e-981a-3d1479aa4792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:27.122494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a4493-12e3-429e-981a-3d1479aa4792 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:27.177461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1211/2401] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:26:27 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:26:27.181637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08f3f5-0d2d-4949-b32c-2cbadf4086a6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:27.193071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08f3f5-0d2d-4949-b32c-2cbadf4086a6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:27.222599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:27.271507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5298851-eb1d-4b83-86d5-977c149dfcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:27.272211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5298851-eb1d-4b83-86d5-977c149dfcf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:27.272559 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5298851-eb1d-4b83-86d5-977c149dfcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:27.272926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5298851-eb1d-4b83-86d5-977c149dfcf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:27.273745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5298851-eb1d-4b83-86d5-977c149dfcf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:27.275859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5298851-eb1d-4b83-86d5-977c149dfcf6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:27.303531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1191/2402] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:26:27 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:26:27.308208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a4493-12e3-429e-981a-3d1479aa4792 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:27.321438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a4493-12e3-429e-981a-3d1479aa4792 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:27.326549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55741dd1-dff6-4c92-99b1-a98e2e3c9aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:27.327225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55741dd1-dff6-4c92-99b1-a98e2e3c9aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:27.327508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55741dd1-dff6-4c92-99b1-a98e2e3c9aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:27.327791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55741dd1-dff6-4c92-99b1-a98e2e3c9aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:27.328695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55741dd1-dff6-4c92-99b1-a98e2e3c9aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:27.330465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55741dd1-dff6-4c92-99b1-a98e2e3c9aa4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:27.342322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1212/2403] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:27.346038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5298851-eb1d-4b83-86d5-977c149dfcf6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:27.358821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5298851-eb1d-4b83-86d5-977c149dfcf6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:27.365961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed9890-bc1e-4f2c-983e-52d5f499adef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:27.366642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed9890-bc1e-4f2c-983e-52d5f499adef admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:27.366926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed9890-bc1e-4f2c-983e-52d5f499adef admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:27.367206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed9890-bc1e-4f2c-983e-52d5f499adef admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:27.367939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed9890-bc1e-4f2c-983e-52d5f499adef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:27.370560 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed9890-bc1e-4f2c-983e-52d5f499adef admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:27.385117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1192/2404] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:26:27.386344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55741dd1-dff6-4c92-99b1-a98e2e3c9aa4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:27.397149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55741dd1-dff6-4c92-99b1-a98e2e3c9aa4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:27.402418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84115875-9d57-435a-891f-b891846f09ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:27.403163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84115875-9d57-435a-891f-b891846f09ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:27.403449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84115875-9d57-435a-891f-b891846f09ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:27.403727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84115875-9d57-435a-891f-b891846f09ae admin admin] PATH_INFO: `/v3/projects/ac52e2c9f1c34c54b3577b9dc1c96aee/users/b8d64d6e211640f895870c444d271c2c/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:27.429166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84115875-9d57-435a-891f-b891846f09ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac52e2c9f1c34c54b3577b9dc1c96aee, user_id=b8d64d6e211640f895870c444d271c2c, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:27.432707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84115875-9d57-435a-891f-b891846f09ae admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:27.450304 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1213/2405] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:27 2022] PUT /identity/v3/projects/ac52e2c9f1c34c54b3577b9dc1c96aee/users/b8d64d6e211640f895870c444d271c2c/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:26:27.461818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e769fa2-82bb-41e1-b4f0-51c2be82aaa1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:27.462184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e769fa2-82bb-41e1-b4f0-51c2be82aaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:27.462466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e769fa2-82bb-41e1-b4f0-51c2be82aaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:27.492500 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6e769fa2-82bb-41e1-b4f0-51c2be82aaa1 None None] MFA Rules not processed for user `b8d64d6e211640f895870c444d271c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:27.493887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6e769fa2-82bb-41e1-b4f0-51c2be82aaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:27.577829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1193/2406] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:27.589486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84115875-9d57-435a-891f-b891846f09ae admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:27.591489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-84115875-9d57-435a-891f-b891846f09ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:27.646079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:27.660270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84115875-9d57-435a-891f-b891846f09ae admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:27.665329 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ad10b-2385-4240-b8ee-74e5f3554f21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:27.666477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ad10b-2385-4240-b8ee-74e5f3554f21 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:27.667338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ad10b-2385-4240-b8ee-74e5f3554f21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:27.667849 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ad10b-2385-4240-b8ee-74e5f3554f21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:27.678234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ad10b-2385-4240-b8ee-74e5f3554f21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:27.679748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ad10b-2385-4240-b8ee-74e5f3554f21 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:27.690687 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1214/2407] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:27.713004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e769fa2-82bb-41e1-b4f0-51c2be82aaa1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:27.780983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e769fa2-82bb-41e1-b4f0-51c2be82aaa1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:27.785616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:27.786773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:27.787055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:27.787326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] PATH_INFO: `/v3/users/b8d64d6e211640f895870c444d271c2c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:27.788508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d64d6e211640f895870c444d271c2c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:27.790478 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:27.834980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:27.849854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d64d6e211640f895870c444d271c2c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:27.858832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d64d6e211640f895870c444d271c2c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:27.859610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d64d6e211640f895870c444d271c2c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:27.873373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d64d6e211640f895870c444d271c2c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:27.880374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d64d6e211640f895870c444d271c2c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:27.889249 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1194/2408] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:26:27 2022] DELETE /identity/v3/users/b8d64d6e211640f895870c444d271c2c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:27.893934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d0210-2bff-44e1-a529-145f075f4726 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:27.894234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d0210-2bff-44e1-a529-145f075f4726 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:27.894512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d0210-2bff-44e1-a529-145f075f4726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:27.912854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bd6d0210-2bff-44e1-a529-145f075f4726 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:27.914087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bd6d0210-2bff-44e1-a529-145f075f4726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:27.986700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1215/2409] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:27.997679 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:27.999156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:28.070367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e69ff-fd3f-47fd-ad8a-79aa858239e2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:28.074113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86355415-74fe-4154-bb6f-3f91bbf8a841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:28.074769 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86355415-74fe-4154-bb6f-3f91bbf8a841 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:28.075054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86355415-74fe-4154-bb6f-3f91bbf8a841 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:28.075331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86355415-74fe-4154-bb6f-3f91bbf8a841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:28.085223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86355415-74fe-4154-bb6f-3f91bbf8a841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:28.086628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86355415-74fe-4154-bb6f-3f91bbf8a841 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:28.105996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1195/2410] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:26:28.484485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d0210-2bff-44e1-a529-145f075f4726 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:28.486203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bd6d0210-2bff-44e1-a529-145f075f4726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:28.521872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d0210-2bff-44e1-a529-145f075f4726 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:28.528250 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:28.529129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:28.529414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:28.529693 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] PATH_INFO: `/v3/projects/ac52e2c9f1c34c54b3577b9dc1c96aee` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:28.530954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac52e2c9f1c34c54b3577b9dc1c96aee)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:28.533136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:28.588771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac52e2c9f1c34c54b3577b9dc1c96aee', 'request_id': 'req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:28.600601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac52e2c9f1c34c54b3577b9dc1c96aee', 'request_id': 'req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:28.612228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac52e2c9f1c34c54b3577b9dc1c96aee', 'request_id': 'req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:28.624037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] The token cache is being invalidate because project ac52e2c9f1c34c54b3577b9dc1c96aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:28.624328 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:28.624974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-018ab9f4-0b40-44c0-9f3f-42d8078e04dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:28.626852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1216/2411] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:28 2022] DELETE /identity/v3/projects/ac52e2c9f1c34c54b3577b9dc1c96aee => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:26:28.632498 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547edbe-39b7-4e23-aaaf-66fa0f59dbec None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:28.633010 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547edbe-39b7-4e23-aaaf-66fa0f59dbec None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:28.633433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547edbe-39b7-4e23-aaaf-66fa0f59dbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:28.654756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9547edbe-39b7-4e23-aaaf-66fa0f59dbec None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:28.655601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9547edbe-39b7-4e23-aaaf-66fa0f59dbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:28.725749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1196/2412] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:28.734223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22275c-7349-455c-8ac1-3350147cddf9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:28.734522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22275c-7349-455c-8ac1-3350147cddf9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:28.734784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22275c-7349-455c-8ac1-3350147cddf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:28.755712 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ba22275c-7349-455c-8ac1-3350147cddf9 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:28.756915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ba22275c-7349-455c-8ac1-3350147cddf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:28.769580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1217/2413] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:28.774033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547edbe-39b7-4e23-aaaf-66fa0f59dbec None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:28.787150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547edbe-39b7-4e23-aaaf-66fa0f59dbec None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:28.791985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91e804-20e8-4beb-8d88-a9df1a1e563b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:28.792888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91e804-20e8-4beb-8d88-a9df1a1e563b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:28.793174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91e804-20e8-4beb-8d88-a9df1a1e563b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:28.793448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91e804-20e8-4beb-8d88-a9df1a1e563b admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:28.794213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91e804-20e8-4beb-8d88-a9df1a1e563b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:26:28.794517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91e804-20e8-4beb-8d88-a9df1a1e563b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:28.796104 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91e804-20e8-4beb-8d88-a9df1a1e563b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:28.810591 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1197/2414] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:26:28.815577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22275c-7349-455c-8ac1-3350147cddf9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:28.829418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22275c-7349-455c-8ac1-3350147cddf9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:28.834395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5361b9b-d064-4ff0-a503-b49a05c81c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:28.835161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5361b9b-d064-4ff0-a503-b49a05c81c44 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:28.835542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5361b9b-d064-4ff0-a503-b49a05c81c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:28.837741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5361b9b-d064-4ff0-a503-b49a05c81c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:28.837741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5361b9b-d064-4ff0-a503-b49a05c81c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:28.839732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5361b9b-d064-4ff0-a503-b49a05c81c44 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:28.892269 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1218/2415] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:26:28 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:28.895614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91e804-20e8-4beb-8d88-a9df1a1e563b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:28.909896 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91e804-20e8-4beb-8d88-a9df1a1e563b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:28.954368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:28.982011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7bb5d-460b-4329-80ee-56c55d1e5427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:28.982823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7bb5d-460b-4329-80ee-56c55d1e5427 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:28.983113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7bb5d-460b-4329-80ee-56c55d1e5427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:28.983395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7bb5d-460b-4329-80ee-56c55d1e5427 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:28.984143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7bb5d-460b-4329-80ee-56c55d1e5427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:28.986228 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7bb5d-460b-4329-80ee-56c55d1e5427 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:29.009829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1198/2416] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:26:28 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:26:29.014189 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5361b9b-d064-4ff0-a503-b49a05c81c44 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:29.027671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5361b9b-d064-4ff0-a503-b49a05c81c44 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:29.032572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfbce7-aad1-4fa1-8814-a9c25a2e458e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:29.033255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfbce7-aad1-4fa1-8814-a9c25a2e458e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:29.033543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfbce7-aad1-4fa1-8814-a9c25a2e458e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:29.033907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfbce7-aad1-4fa1-8814-a9c25a2e458e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:29.034647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfbce7-aad1-4fa1-8814-a9c25a2e458e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:29.036258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfbce7-aad1-4fa1-8814-a9c25a2e458e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:29.051630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1219/2417] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:29.053034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7bb5d-460b-4329-80ee-56c55d1e5427 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:29.065391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7bb5d-460b-4329-80ee-56c55d1e5427 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:29.070861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b93d0e-911a-4bbb-84e8-506a2b445c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:29.071776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b93d0e-911a-4bbb-84e8-506a2b445c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:29.072213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b93d0e-911a-4bbb-84e8-506a2b445c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:29.072333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b93d0e-911a-4bbb-84e8-506a2b445c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:29.073335 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b93d0e-911a-4bbb-84e8-506a2b445c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:29.074988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b93d0e-911a-4bbb-84e8-506a2b445c9f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:29.086817 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1199/2418] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:26:29.092066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfbce7-aad1-4fa1-8814-a9c25a2e458e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:29.103381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfbce7-aad1-4fa1-8814-a9c25a2e458e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:29.111593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c6ddc-d973-48f8-8464-93d046edf01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:29.112493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c6ddc-d973-48f8-8464-93d046edf01e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:29.112774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c6ddc-d973-48f8-8464-93d046edf01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:29.113047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c6ddc-d973-48f8-8464-93d046edf01e admin admin] PATH_INFO: `/v3/projects/c5133fe108f54ff3bdea9540b9f9e46d/users/e6bd8568fc884fceb5ee60a7ab74c27e/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:29.137048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c6ddc-d973-48f8-8464-93d046edf01e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5133fe108f54ff3bdea9540b9f9e46d, user_id=e6bd8568fc884fceb5ee60a7ab74c27e, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:29.140294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c6ddc-d973-48f8-8464-93d046edf01e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:29.152974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:29.166848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1220/2419] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:26:29 2022] PUT /identity/v3/projects/c5133fe108f54ff3bdea9540b9f9e46d/users/e6bd8568fc884fceb5ee60a7ab74c27e/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:29.177895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220d2db-4747-4427-aecf-71631767d047 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:29.178757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220d2db-4747-4427-aecf-71631767d047 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:29.179117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220d2db-4747-4427-aecf-71631767d047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:29.214147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5220d2db-4747-4427-aecf-71631767d047 None None] MFA Rules not processed for user `e6bd8568fc884fceb5ee60a7ab74c27e`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:29.215394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5220d2db-4747-4427-aecf-71631767d047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:29.302900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1200/2420] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:29.314241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c6ddc-d973-48f8-8464-93d046edf01e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:29.316190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cc0c6ddc-d973-48f8-8464-93d046edf01e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:29.386607 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c6ddc-d973-48f8-8464-93d046edf01e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:29.392352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17414ce-b936-415b-bea9-78eedde35e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:29.395233 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17414ce-b936-415b-bea9-78eedde35e12 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:29.396238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17414ce-b936-415b-bea9-78eedde35e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:29.396788 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17414ce-b936-415b-bea9-78eedde35e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:29.407063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17414ce-b936-415b-bea9-78eedde35e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:29.408938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17414ce-b936-415b-bea9-78eedde35e12 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:29.419566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1221/2421] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:29.453125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220d2db-4747-4427-aecf-71631767d047 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:29.518841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220d2db-4747-4427-aecf-71631767d047 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:29.523557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:29.524446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:29.524732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:29.525012 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] PATH_INFO: `/v3/users/e6bd8568fc884fceb5ee60a7ab74c27e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:29.529527 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6bd8568fc884fceb5ee60a7ab74c27e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:29.529836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:29.589643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6bd8568fc884fceb5ee60a7ab74c27e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:29.594959 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:29.599582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6bd8568fc884fceb5ee60a7ab74c27e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:29.600800 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6bd8568fc884fceb5ee60a7ab74c27e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:29.628691 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6bd8568fc884fceb5ee60a7ab74c27e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:29.640282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6bd8568fc884fceb5ee60a7ab74c27e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:29.653954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1201/2422] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:26:29 2022] DELETE /identity/v3/users/e6bd8568fc884fceb5ee60a7ab74c27e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:29.654918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18012-1d66-4c75-b695-db7a89b02c5e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:29.655502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18012-1d66-4c75-b695-db7a89b02c5e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:29.655606 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18012-1d66-4c75-b695-db7a89b02c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:29.676112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cea18012-1d66-4c75-b695-db7a89b02c5e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:29.677476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cea18012-1d66-4c75-b695-db7a89b02c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:29.753728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1222/2423] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:29.761212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:29.763039 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:29.836899 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c609c85-6ee6-4784-a91e-ab47a399d372 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:29.841277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4448504-9171-47dc-a645-16cf61d93e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:29.841967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4448504-9171-47dc-a645-16cf61d93e3d service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:29.842262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4448504-9171-47dc-a645-16cf61d93e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:29.842535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4448504-9171-47dc-a645-16cf61d93e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:29.854909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4448504-9171-47dc-a645-16cf61d93e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:29.856354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4448504-9171-47dc-a645-16cf61d93e3d service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:29.867445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1202/2424] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:30.192513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18012-1d66-4c75-b695-db7a89b02c5e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:30.194126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cea18012-1d66-4c75-b695-db7a89b02c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:30.229561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18012-1d66-4c75-b695-db7a89b02c5e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:30.249434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:30.251761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:30.252416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:30.252893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] PATH_INFO: `/v3/projects/c5133fe108f54ff3bdea9540b9f9e46d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:30.254388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5133fe108f54ff3bdea9540b9f9e46d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:30.257312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:30.324656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5133fe108f54ff3bdea9540b9f9e46d', 'request_id': 'req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:30.330903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5133fe108f54ff3bdea9540b9f9e46d', 'request_id': 'req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:30.340443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5133fe108f54ff3bdea9540b9f9e46d', 'request_id': 'req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:30.355173 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] The token cache is being invalidate because project c5133fe108f54ff3bdea9540b9f9e46d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:26:30.355478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:30.356844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:26:30.358361 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1223/2425] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:26:30 2022] DELETE /identity/v3/projects/c5133fe108f54ff3bdea9540b9f9e46d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:26:39.861900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=3,keystone:SELECT=38 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:40.349568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:40.655432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45caf6-09cd-492e-881a-e57be408140b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:40.655920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45caf6-09cd-492e-881a-e57be408140b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:40.656423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45caf6-09cd-492e-881a-e57be408140b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:40.675536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2b45caf6-09cd-492e-881a-e57be408140b None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:40.676659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2b45caf6-09cd-492e-881a-e57be408140b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:40.736673 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1203/2426] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:26:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:40.749152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:40.750600 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:40.818782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabcf1d-a3a3-4b96-b3d8-41b53e8618ac admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:40.823806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5d630-4519-4a6d-8008-11c30f7b262a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:40.824448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d630-4519-4a6d-8008-11c30f7b262a service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:40.824736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d630-4519-4a6d-8008-11c30f7b262a service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:40.825016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d630-4519-4a6d-8008-11c30f7b262a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:40.833252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5d630-4519-4a6d-8008-11c30f7b262a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:40.835691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5d630-4519-4a6d-8008-11c30f7b262a service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:40.843033 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1224/2427] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:26:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:26:50.729587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=25 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:50.839718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=32 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:58.626274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd12b5-b865-4d3b-9b9c-8e7b7cff8f2c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:58.626770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd12b5-b865-4d3b-9b9c-8e7b7cff8f2c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:58.626876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd12b5-b865-4d3b-9b9c-8e7b7cff8f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:58.645116 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-34fd12b5-b865-4d3b-9b9c-8e7b7cff8f2c None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:26:58.646977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-34fd12b5-b865-4d3b-9b9c-8e7b7cff8f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:58.654750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1204/2428] 10.222.0.42 () {58 vars in 967 bytes} [Wed May 25 16:26:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:26:58.668508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5d630-4519-4a6d-8008-11c30f7b262a service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:58.679002 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5d630-4519-4a6d-8008-11c30f7b262a service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:58.684005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333075b1-5e14-4210-bca0-d8e7570dc57f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:58.684625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333075b1-5e14-4210-bca0-d8e7570dc57f service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:58.684917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333075b1-5e14-4210-bca0-d8e7570dc57f service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:58.685197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333075b1-5e14-4210-bca0-d8e7570dc57f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:58.693238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333075b1-5e14-4210-bca0-d8e7570dc57f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:58.694867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333075b1-5e14-4210-bca0-d8e7570dc57f service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:58.703968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1225/2429] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:26:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:27:08.639783 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:08.700553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:29.450453 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd12b5-b865-4d3b-9b9c-8e7b7cff8f2c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:29.451972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-34fd12b5-b865-4d3b-9b9c-8e7b7cff8f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:29.514313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd12b5-b865-4d3b-9b9c-8e7b7cff8f2c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:29.518532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9756cd5-011a-4a67-9f30-46ffcab41c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:29.519255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756cd5-011a-4a67-9f30-46ffcab41c0f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:29.519541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756cd5-011a-4a67-9f30-46ffcab41c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:29.519815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756cd5-011a-4a67-9f30-46ffcab41c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:29.521611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a9756cd5-011a-4a67-9f30-46ffcab41c0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:29.592364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9756cd5-011a-4a67-9f30-46ffcab41c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:29.594005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9756cd5-011a-4a67-9f30-46ffcab41c0f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:29.603909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1205/2430] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:29.621387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333075b1-5e14-4210-bca0-d8e7570dc57f service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:29.622879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-333075b1-5e14-4210-bca0-d8e7570dc57f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:29.635039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333075b1-5e14-4210-bca0-d8e7570dc57f service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:29.638872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bc740-fe52-464e-810d-5d503d7d3a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:29.639508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bc740-fe52-464e-810d-5d503d7d3a8d service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:29.639802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bc740-fe52-464e-810d-5d503d7d3a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:29.640084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bc740-fe52-464e-810d-5d503d7d3a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:29.641841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ba9bc740-fe52-464e-810d-5d503d7d3a8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:29.653696 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bc740-fe52-464e-810d-5d503d7d3a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:29.655160 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bc740-fe52-464e-810d-5d503d7d3a8d service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:29.665195 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1226/2431] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:33.146789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9756cd5-011a-4a67-9f30-46ffcab41c0f service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:33.148348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a9756cd5-011a-4a67-9f30-46ffcab41c0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:33.227524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bc740-fe52-464e-810d-5d503d7d3a8d service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:33.230537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ba9bc740-fe52-464e-810d-5d503d7d3a8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:33.255965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bc740-fe52-464e-810d-5d503d7d3a8d service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:33.265014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:33.266015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:33.266663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:33.267178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] PATH_INFO: `/v3/users/15fae427f22d476fa0e378949de41292` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:33.268737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15fae427f22d476fa0e378949de41292)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:33.272398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:33.288254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9756cd5-011a-4a67-9f30-46ffcab41c0f service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:33.294775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e855fc-5921-4b41-9655-2dd63cb50ae8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:33.295520 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e855fc-5921-4b41-9655-2dd63cb50ae8 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:33.295881 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e855fc-5921-4b41-9655-2dd63cb50ae8 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:33.296372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e855fc-5921-4b41-9655-2dd63cb50ae8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:33.298378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-31e855fc-5921-4b41-9655-2dd63cb50ae8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:33.312534 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e855fc-5921-4b41-9655-2dd63cb50ae8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:33.314833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e855fc-5921-4b41-9655-2dd63cb50ae8 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:33.339173 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1206/2432] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:33.365664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15fae427f22d476fa0e378949de41292'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.367078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15fae427f22d476fa0e378949de41292'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.367948 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15fae427f22d476fa0e378949de41292'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.394313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15fae427f22d476fa0e378949de41292'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.395265 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e855fc-5921-4b41-9655-2dd63cb50ae8 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:33.405800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15fae427f22d476fa0e378949de41292'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.409297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e855fc-5921-4b41-9655-2dd63cb50ae8 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:33.417369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1328f3e-a406-4e2e-80d6-9666efb0ca1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:33.418274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1328f3e-a406-4e2e-80d6-9666efb0ca1b service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:33.418666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1328f3e-a406-4e2e-80d6-9666efb0ca1b service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:33.419343 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1328f3e-a406-4e2e-80d6-9666efb0ca1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:33.421563 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f1328f3e-a406-4e2e-80d6-9666efb0ca1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:33.430290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1227/2433] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:33 2022] DELETE /identity/v3/users/15fae427f22d476fa0e378949de41292 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:33.438550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1328f3e-a406-4e2e-80d6-9666efb0ca1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:33.440399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1328f3e-a406-4e2e-80d6-9666efb0ca1b service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:33.452001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1207/2434] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:33.513890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:33.519164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:33.628982 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f6004-52d0-4034-a69e-866638a34cf5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:33.634549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529dceae-cba0-4038-94ba-5e1690987ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:33.636318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529dceae-cba0-4038-94ba-5e1690987ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:33.636603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529dceae-cba0-4038-94ba-5e1690987ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:33.636880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529dceae-cba0-4038-94ba-5e1690987ae2 admin admin] PATH_INFO: `/v3/users/b781c4c6cb61439fb781988a7da028ff` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:33.638154 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529dceae-cba0-4038-94ba-5e1690987ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b781c4c6cb61439fb781988a7da028ff)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:33.640668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529dceae-cba0-4038-94ba-5e1690987ae2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:33.687258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1328f3e-a406-4e2e-80d6-9666efb0ca1b service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:33.689263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f1328f3e-a406-4e2e-80d6-9666efb0ca1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:33.703336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-529dceae-cba0-4038-94ba-5e1690987ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b781c4c6cb61439fb781988a7da028ff'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.704103 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-529dceae-cba0-4038-94ba-5e1690987ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b781c4c6cb61439fb781988a7da028ff'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.704821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-529dceae-cba0-4038-94ba-5e1690987ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b781c4c6cb61439fb781988a7da028ff'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.719648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:33.720921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-529dceae-cba0-4038-94ba-5e1690987ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b781c4c6cb61439fb781988a7da028ff'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.724303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1328f3e-a406-4e2e-80d6-9666efb0ca1b service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:33.731642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-529dceae-cba0-4038-94ba-5e1690987ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b781c4c6cb61439fb781988a7da028ff'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.731818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:33.732560 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:33.732849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:33.733243 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] PATH_INFO: `/v3/projects/3e8c39550e4e41e79cc56aa7d22ae052` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:33.734463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e8c39550e4e41e79cc56aa7d22ae052)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:33.736592 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:33.747974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1228/2435] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:33 2022] DELETE /identity/v3/users/b781c4c6cb61439fb781988a7da028ff => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:33.769999 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:33.799851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e8c39550e4e41e79cc56aa7d22ae052', 'request_id': 'req-0dc60a66-b830-4559-9950-adb254083c32'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.806824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e8c39550e4e41e79cc56aa7d22ae052', 'request_id': 'req-0dc60a66-b830-4559-9950-adb254083c32'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.814446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e8c39550e4e41e79cc56aa7d22ae052', 'request_id': 'req-0dc60a66-b830-4559-9950-adb254083c32'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.830646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] The token cache is being invalidate because project 3e8c39550e4e41e79cc56aa7d22ae052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:33.831051 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.831788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:33.833743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1208/2436] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:33 2022] DELETE /identity/v3/projects/3e8c39550e4e41e79cc56aa7d22ae052 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:33.842166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0807fb-ffe4-47e3-88e3-a4cf5b961212 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:33.842677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0807fb-ffe4-47e3-88e3-a4cf5b961212 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:33.842961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0807fb-ffe4-47e3-88e3-a4cf5b961212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:33.870784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fa0807fb-ffe4-47e3-88e3-a4cf5b961212 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:33.871697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fa0807fb-ffe4-47e3-88e3-a4cf5b961212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:33.960208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:33.961754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:33.963271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1229/2437] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:33.965341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4669f4-0a04-40f7-8bb4-f8d1df07f3a9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:33.965661 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4669f4-0a04-40f7-8bb4-f8d1df07f3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:33.966066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4669f4-0a04-40f7-8bb4-f8d1df07f3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:33.990134 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc60a66-b830-4559-9950-adb254083c32 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:33.994891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ea4669f4-0a04-40f7-8bb4-f8d1df07f3a9 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:33.995905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:33.996161 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ea4669f4-0a04-40f7-8bb4-f8d1df07f3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:33.996968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:33.997257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:33.997539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] PATH_INFO: `/v3/projects/9d234edace28441282f594f0e03cacc5` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:33.998813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d234edace28441282f594f0e03cacc5)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.000741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.005106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1230/2438] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:34.009837 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4669f4-0a04-40f7-8bb4-f8d1df07f3a9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.034480 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4669f4-0a04-40f7-8bb4-f8d1df07f3a9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.040880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cc4a8-3bb3-45df-ba06-7df2930e7e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.041971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cc4a8-3bb3-45df-ba06-7df2930e7e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.042404 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cc4a8-3bb3-45df-ba06-7df2930e7e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.042825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cc4a8-3bb3-45df-ba06-7df2930e7e3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.043880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cc4a8-3bb3-45df-ba06-7df2930e7e3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:27:34.045922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cc4a8-3bb3-45df-ba06-7df2930e7e3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.048614 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cc4a8-3bb3-45df-ba06-7df2930e7e3f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.064257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1231/2439] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:27:34.068378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cc4a8-3bb3-45df-ba06-7df2930e7e3f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.070588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d234edace28441282f594f0e03cacc5', 'request_id': 'req-7393734e-bfa0-44b8-a289-fbb99446ef69'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:34.077445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d234edace28441282f594f0e03cacc5', 'request_id': 'req-7393734e-bfa0-44b8-a289-fbb99446ef69'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:34.082011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cc4a8-3bb3-45df-ba06-7df2930e7e3f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.086142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d234edace28441282f594f0e03cacc5', 'request_id': 'req-7393734e-bfa0-44b8-a289-fbb99446ef69'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:34.108909 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] The token cache is being invalidate because project 9d234edace28441282f594f0e03cacc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:34.109350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:34.111081 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7393734e-bfa0-44b8-a289-fbb99446ef69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:34.113075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1209/2440] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:33 2022] DELETE /identity/v3/projects/9d234edace28441282f594f0e03cacc5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:27:34.117960 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382f03e-fabe-411b-9588-d12ea63f84de None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.118395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382f03e-fabe-411b-9588-d12ea63f84de None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.118789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382f03e-fabe-411b-9588-d12ea63f84de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.143367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7382f03e-fabe-411b-9588-d12ea63f84de None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:34.144596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7382f03e-fabe-411b-9588-d12ea63f84de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:34.169744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536b53f-df94-4383-a6dd-5e32458479cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.170576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536b53f-df94-4383-a6dd-5e32458479cd admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.170932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536b53f-df94-4383-a6dd-5e32458479cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.171276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536b53f-df94-4383-a6dd-5e32458479cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.172066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536b53f-df94-4383-a6dd-5e32458479cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.177617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536b53f-df94-4383-a6dd-5e32458479cd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.205885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1232/2441] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:34 2022] POST /identity/v3/projects => generated 363 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:27:34.209565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536b53f-df94-4383-a6dd-5e32458479cd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.212318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f536b53f-df94-4383-a6dd-5e32458479cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:34.231137 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1210/2442] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:34.239886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f24608-abf0-438f-a87f-6c0dd77d8533 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.240179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f24608-abf0-438f-a87f-6c0dd77d8533 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.240485 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f24608-abf0-438f-a87f-6c0dd77d8533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.261596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c2f24608-abf0-438f-a87f-6c0dd77d8533 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:34.262873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c2f24608-abf0-438f-a87f-6c0dd77d8533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:34.275311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:34.297441 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:34.311240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536b53f-df94-4383-a6dd-5e32458479cd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.316180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de1777-efc0-45ce-a3df-f2afec7c233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.316857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de1777-efc0-45ce-a3df-f2afec7c233e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.317323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de1777-efc0-45ce-a3df-f2afec7c233e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.319143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de1777-efc0-45ce-a3df-f2afec7c233e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.320012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de1777-efc0-45ce-a3df-f2afec7c233e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.322043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de1777-efc0-45ce-a3df-f2afec7c233e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.349590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1233/2443] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:34 2022] POST /identity/v3/users => generated 282 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:34.350852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de1777-efc0-45ce-a3df-f2afec7c233e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.351400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1211/2444] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:27:34.355833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f24608-abf0-438f-a87f-6c0dd77d8533 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.362209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de1777-efc0-45ce-a3df-f2afec7c233e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.366918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f24608-abf0-438f-a87f-6c0dd77d8533 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.367525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a57027-e2f8-4f80-ae53-e92899576a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.368228 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a57027-e2f8-4f80-ae53-e92899576a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.368515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a57027-e2f8-4f80-ae53-e92899576a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.368796 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a57027-e2f8-4f80-ae53-e92899576a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.370205 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a57027-e2f8-4f80-ae53-e92899576a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.371823 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a57027-e2f8-4f80-ae53-e92899576a19 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.373263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71a09e-6316-4e5d-9094-b5f3a745abb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.376222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71a09e-6316-4e5d-9094-b5f3a745abb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.376515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71a09e-6316-4e5d-9094-b5f3a745abb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.376791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71a09e-6316-4e5d-9094-b5f3a745abb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.379584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71a09e-6316-4e5d-9094-b5f3a745abb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:27:34.381544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71a09e-6316-4e5d-9094-b5f3a745abb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.383191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71a09e-6316-4e5d-9094-b5f3a745abb0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.383756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1234/2445] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:34.388352 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a57027-e2f8-4f80-ae53-e92899576a19 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.398983 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1212/2446] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:27:34.403631 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71a09e-6316-4e5d-9094-b5f3a745abb0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.405118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a57027-e2f8-4f80-ae53-e92899576a19 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.414032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e21574-7592-490a-9680-940998ea33f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.414358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71a09e-6316-4e5d-9094-b5f3a745abb0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.414710 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e21574-7592-490a-9680-940998ea33f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.414996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e21574-7592-490a-9680-940998ea33f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.415274 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e21574-7592-490a-9680-940998ea33f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.416011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e21574-7592-490a-9680-940998ea33f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.417842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e21574-7592-490a-9680-940998ea33f5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.420598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069932cc-e197-4f1e-8531-95a2636e08ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.421390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069932cc-e197-4f1e-8531-95a2636e08ea admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.421677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069932cc-e197-4f1e-8531-95a2636e08ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.422007 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069932cc-e197-4f1e-8531-95a2636e08ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.422690 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069932cc-e197-4f1e-8531-95a2636e08ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.424587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069932cc-e197-4f1e-8531-95a2636e08ea admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.429394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1235/2447] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:27:34.434449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e21574-7592-490a-9680-940998ea33f5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.444378 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e21574-7592-490a-9680-940998ea33f5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.450318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acb7f2-baa1-405d-89ba-70fc5200f175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.451466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acb7f2-baa1-405d-89ba-70fc5200f175 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.451592 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acb7f2-baa1-405d-89ba-70fc5200f175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.451798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acb7f2-baa1-405d-89ba-70fc5200f175 admin admin] PATH_INFO: `/v3/projects/275be89764304e3aaea606fd8db79af1/users/1ad2312ac7694e608b9557a12d73e30c/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.455428 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1213/2448] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:34 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:34.459451 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069932cc-e197-4f1e-8531-95a2636e08ea admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.467339 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acb7f2-baa1-405d-89ba-70fc5200f175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=275be89764304e3aaea606fd8db79af1, user_id=1ad2312ac7694e608b9557a12d73e30c, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.470100 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acb7f2-baa1-405d-89ba-70fc5200f175 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.483369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1236/2449] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:34 2022] PUT /identity/v3/projects/275be89764304e3aaea606fd8db79af1/users/1ad2312ac7694e608b9557a12d73e30c/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:27:34.495310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25091996-8dd8-4209-8226-59e6202d21be None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.495810 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25091996-8dd8-4209-8226-59e6202d21be None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.496407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25091996-8dd8-4209-8226-59e6202d21be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.522829 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-25091996-8dd8-4209-8226-59e6202d21be None None] MFA Rules not processed for user `1ad2312ac7694e608b9557a12d73e30c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:34.524042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-25091996-8dd8-4209-8226-59e6202d21be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:34.527962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069932cc-e197-4f1e-8531-95a2636e08ea admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.585957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0fd09-1d8f-4d26-a66c-bffe34a0e18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.586656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0fd09-1d8f-4d26-a66c-bffe34a0e18f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.586942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0fd09-1d8f-4d26-a66c-bffe34a0e18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.587250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0fd09-1d8f-4d26-a66c-bffe34a0e18f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.587937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0fd09-1d8f-4d26-a66c-bffe34a0e18f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.589700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1237/2450] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:34.595309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0fd09-1d8f-4d26-a66c-bffe34a0e18f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.603031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25091996-8dd8-4209-8226-59e6202d21be None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.605412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-25091996-8dd8-4209-8226-59e6202d21be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:34.615187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1214/2451] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:34 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:27:34.619336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0fd09-1d8f-4d26-a66c-bffe34a0e18f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.629860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0fd09-1d8f-4d26-a66c-bffe34a0e18f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.634742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a6003-c64f-4803-af52-f92101755c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.635574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a6003-c64f-4803-af52-f92101755c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.635857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a6003-c64f-4803-af52-f92101755c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.636325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a6003-c64f-4803-af52-f92101755c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.637060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a6003-c64f-4803-af52-f92101755c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.638699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a6003-c64f-4803-af52-f92101755c83 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.648891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1215/2452] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:27:34.653164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a6003-c64f-4803-af52-f92101755c83 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.663827 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a6003-c64f-4803-af52-f92101755c83 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.669872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d9d5b-6a06-492b-81fd-5fa124828919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.670659 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d9d5b-6a06-492b-81fd-5fa124828919 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.670952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d9d5b-6a06-492b-81fd-5fa124828919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.671232 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d9d5b-6a06-492b-81fd-5fa124828919 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.671956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d9d5b-6a06-492b-81fd-5fa124828919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.673608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25091996-8dd8-4209-8226-59e6202d21be None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.673755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d9d5b-6a06-492b-81fd-5fa124828919 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.677740 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde7163-7698-489f-9506-080b6e4a382b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.678403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde7163-7698-489f-9506-080b6e4a382b service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.678705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde7163-7698-489f-9506-080b6e4a382b service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.678980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde7163-7698-489f-9506-080b6e4a382b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.685565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1216/2453] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:27:34.689673 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d9d5b-6a06-492b-81fd-5fa124828919 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.691825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde7163-7698-489f-9506-080b6e4a382b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.693385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde7163-7698-489f-9506-080b6e4a382b service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.699741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d9d5b-6a06-492b-81fd-5fa124828919 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.703117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1238/2454] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:34 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:27:34.704537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984bb8a-20a2-42cf-9f33-c9dd6f7b9e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.705391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984bb8a-20a2-42cf-9f33-c9dd6f7b9e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.705673 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984bb8a-20a2-42cf-9f33-c9dd6f7b9e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.705988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984bb8a-20a2-42cf-9f33-c9dd6f7b9e12 admin admin] PATH_INFO: `/v3/projects/e7f882a220d24bd5b295deceac26280e/users/5bc4fde81b7645b6aa0b5c1f2c28d313/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.717613 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:34.722377 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984bb8a-20a2-42cf-9f33-c9dd6f7b9e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f882a220d24bd5b295deceac26280e, user_id=5bc4fde81b7645b6aa0b5c1f2c28d313, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.725794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984bb8a-20a2-42cf-9f33-c9dd6f7b9e12 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.739599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1217/2455] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:34 2022] PUT /identity/v3/projects/e7f882a220d24bd5b295deceac26280e/users/5bc4fde81b7645b6aa0b5c1f2c28d313/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:27:34.751184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a945-2680-417a-ba35-6e34b08e5c5e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.751697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a945-2680-417a-ba35-6e34b08e5c5e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.752057 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a945-2680-417a-ba35-6e34b08e5c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.780694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b8f4a945-2680-417a-ba35-6e34b08e5c5e None None] MFA Rules not processed for user `5bc4fde81b7645b6aa0b5c1f2c28d313`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:34.781952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b8f4a945-2680-417a-ba35-6e34b08e5c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:34.790400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:34.855150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1239/2456] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:34.860162 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984bb8a-20a2-42cf-9f33-c9dd6f7b9e12 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.926842 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984bb8a-20a2-42cf-9f33-c9dd6f7b9e12 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:34.931615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1fe1b-16b6-41df-91ae-5c46582afd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:34.933153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1fe1b-16b6-41df-91ae-5c46582afd7a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:34.933589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1fe1b-16b6-41df-91ae-5c46582afd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:34.933986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1fe1b-16b6-41df-91ae-5c46582afd7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:34.934837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1fe1b-16b6-41df-91ae-5c46582afd7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:34.936945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1fe1b-16b6-41df-91ae-5c46582afd7a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:34.968863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1218/2457] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:34 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:27:34.972812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a945-2680-417a-ba35-6e34b08e5c5e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:34.987045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a945-2680-417a-ba35-6e34b08e5c5e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:35.055323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8db9ff-c843-401e-91db-98bb64dc2765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:35.056015 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8db9ff-c843-401e-91db-98bb64dc2765 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.056295 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8db9ff-c843-401e-91db-98bb64dc2765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.056571 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8db9ff-c843-401e-91db-98bb64dc2765 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.058037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8db9ff-c843-401e-91db-98bb64dc2765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:35.060046 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8db9ff-c843-401e-91db-98bb64dc2765 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:35.079574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1240/2458] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:34 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:35.083698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1fe1b-16b6-41df-91ae-5c46582afd7a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:35.094350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1fe1b-16b6-41df-91ae-5c46582afd7a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:35.098935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bcb8e-63a4-4729-bf5d-5346e19fa859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:35.099587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bcb8e-63a4-4729-bf5d-5346e19fa859 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.099876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bcb8e-63a4-4729-bf5d-5346e19fa859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.100163 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bcb8e-63a4-4729-bf5d-5346e19fa859 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.100872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bcb8e-63a4-4729-bf5d-5346e19fa859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:35.102495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bcb8e-63a4-4729-bf5d-5346e19fa859 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:35.116412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1219/2459] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:35.121210 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8db9ff-c843-401e-91db-98bb64dc2765 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:35.131048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8db9ff-c843-401e-91db-98bb64dc2765 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:35.136543 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66734d-05e9-4d7f-a443-425af89e8fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:35.137197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66734d-05e9-4d7f-a443-425af89e8fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.137483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66734d-05e9-4d7f-a443-425af89e8fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.137822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66734d-05e9-4d7f-a443-425af89e8fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.138554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66734d-05e9-4d7f-a443-425af89e8fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:35.140144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66734d-05e9-4d7f-a443-425af89e8fb9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:35.153619 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1241/2460] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:35.154777 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bcb8e-63a4-4729-bf5d-5346e19fa859 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:35.164914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bcb8e-63a4-4729-bf5d-5346e19fa859 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:35.170218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d130d33-f19f-40f2-afc7-a96f536c644a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:35.171086 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d130d33-f19f-40f2-afc7-a96f536c644a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.171364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d130d33-f19f-40f2-afc7-a96f536c644a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.171637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d130d33-f19f-40f2-afc7-a96f536c644a admin admin] PATH_INFO: `/v3/projects/b624e96fb3354bdc8220c2b367ab4946/users/399047e53fd84357a523409113d8e701/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.187985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d130d33-f19f-40f2-afc7-a96f536c644a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b624e96fb3354bdc8220c2b367ab4946, user_id=399047e53fd84357a523409113d8e701, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:35.190695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d130d33-f19f-40f2-afc7-a96f536c644a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:35.207164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1220/2461] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:35 2022] PUT /identity/v3/projects/b624e96fb3354bdc8220c2b367ab4946/users/399047e53fd84357a523409113d8e701/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:27:35.219151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388c574-223f-465d-9be6-8e67d16c6b43 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.219342 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388c574-223f-465d-9be6-8e67d16c6b43 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.219617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388c574-223f-465d-9be6-8e67d16c6b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.247870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b388c574-223f-465d-9be6-8e67d16c6b43 None None] MFA Rules not processed for user `399047e53fd84357a523409113d8e701`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:35.249126 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b388c574-223f-465d-9be6-8e67d16c6b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:35.324146 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1242/2462] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:35.328664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d130d33-f19f-40f2-afc7-a96f536c644a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:35.407279 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d130d33-f19f-40f2-afc7-a96f536c644a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:35.412454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668da85-74da-4d68-9ffb-baf850a53942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:35.413119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668da85-74da-4d68-9ffb-baf850a53942 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.413429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668da85-74da-4d68-9ffb-baf850a53942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.413701 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668da85-74da-4d68-9ffb-baf850a53942 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.414457 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668da85-74da-4d68-9ffb-baf850a53942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:35.416926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668da85-74da-4d68-9ffb-baf850a53942 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:35.447702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1221/2463] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:35 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:27:35.448834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388c574-223f-465d-9be6-8e67d16c6b43 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:35.460489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388c574-223f-465d-9be6-8e67d16c6b43 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:35.473135 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:35.520200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c97625-6dea-4a4b-a7fc-5a83721aac95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:35.520993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c97625-6dea-4a4b-a7fc-5a83721aac95 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.521277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c97625-6dea-4a4b-a7fc-5a83721aac95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.521578 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c97625-6dea-4a4b-a7fc-5a83721aac95 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.522297 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c97625-6dea-4a4b-a7fc-5a83721aac95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:35.524214 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c97625-6dea-4a4b-a7fc-5a83721aac95 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:35.553450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1243/2464] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:35 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:35.554621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668da85-74da-4d68-9ffb-baf850a53942 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:35.566864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668da85-74da-4d68-9ffb-baf850a53942 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:35.571930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c561dd-6419-49a7-bf72-e0eff84a6ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:35.572661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c561dd-6419-49a7-bf72-e0eff84a6ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.573023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c561dd-6419-49a7-bf72-e0eff84a6ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.573298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c561dd-6419-49a7-bf72-e0eff84a6ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.574066 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c561dd-6419-49a7-bf72-e0eff84a6ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:35.575711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c561dd-6419-49a7-bf72-e0eff84a6ff0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:35.588118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1222/2465] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:35.592758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c97625-6dea-4a4b-a7fc-5a83721aac95 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:35.603688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c97625-6dea-4a4b-a7fc-5a83721aac95 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:35.610644 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa12007-1529-42ac-a839-45321e38e871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:35.611949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa12007-1529-42ac-a839-45321e38e871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.612238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa12007-1529-42ac-a839-45321e38e871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.612521 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa12007-1529-42ac-a839-45321e38e871 admin admin] PATH_INFO: `/v3/projects/5665680c7ec74be9b58a42a3f14db531/users/8be5803e7a1d4114a58f578a1ae1f500/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.635050 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa12007-1529-42ac-a839-45321e38e871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5665680c7ec74be9b58a42a3f14db531, user_id=8be5803e7a1d4114a58f578a1ae1f500, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:35.639416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa12007-1529-42ac-a839-45321e38e871 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:35.662593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1244/2466] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:35 2022] PUT /identity/v3/projects/5665680c7ec74be9b58a42a3f14db531/users/8be5803e7a1d4114a58f578a1ae1f500/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:27:35.667530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c561dd-6419-49a7-bf72-e0eff84a6ff0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:35.679541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c561dd-6419-49a7-bf72-e0eff84a6ff0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:35.691354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:35.748092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d767608-b3af-46a8-9441-46399ec6ae5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:35.748810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d767608-b3af-46a8-9441-46399ec6ae5c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.749161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d767608-b3af-46a8-9441-46399ec6ae5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.749506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d767608-b3af-46a8-9441-46399ec6ae5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.750333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d767608-b3af-46a8-9441-46399ec6ae5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:35.752039 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d767608-b3af-46a8-9441-46399ec6ae5c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:35.765248 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1223/2467] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:35.767403 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa12007-1529-42ac-a839-45321e38e871 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:35.778225 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa12007-1529-42ac-a839-45321e38e871 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:35.783048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768be86-cff9-405e-b2b2-0885f6973a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:35.784003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768be86-cff9-405e-b2b2-0885f6973a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.784360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768be86-cff9-405e-b2b2-0885f6973a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.784708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768be86-cff9-405e-b2b2-0885f6973a14 admin admin] PATH_INFO: `/v3/projects/5665680c7ec74be9b58a42a3f14db531/users/8be5803e7a1d4114a58f578a1ae1f500/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.787051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768be86-cff9-405e-b2b2-0885f6973a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5665680c7ec74be9b58a42a3f14db531, user_id=8be5803e7a1d4114a58f578a1ae1f500, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:35.789820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768be86-cff9-405e-b2b2-0885f6973a14 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:35.801100 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1245/2468] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:35 2022] PUT /identity/v3/projects/5665680c7ec74be9b58a42a3f14db531/users/8be5803e7a1d4114a58f578a1ae1f500/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:35.812212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e37e2-a3e2-4dc5-a932-3492d2d2694b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.812507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e37e2-a3e2-4dc5-a932-3492d2d2694b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.812789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e37e2-a3e2-4dc5-a932-3492d2d2694b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.840136 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-be6e37e2-a3e2-4dc5-a932-3492d2d2694b None None] MFA Rules not processed for user `8be5803e7a1d4114a58f578a1ae1f500`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:35.841358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-be6e37e2-a3e2-4dc5-a932-3492d2d2694b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:35.912718 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1224/2469] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:35.924110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768be86-cff9-405e-b2b2-0885f6973a14 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:35.983856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768be86-cff9-405e-b2b2-0885f6973a14 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:35.987534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b80a96-8baa-42d9-834b-825cc9d31b91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:35.988177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b80a96-8baa-42d9-834b-825cc9d31b91 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:35.988458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b80a96-8baa-42d9-834b-825cc9d31b91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:35.988730 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b80a96-8baa-42d9-834b-825cc9d31b91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:35.999181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b80a96-8baa-42d9-834b-825cc9d31b91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:36.000591 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b80a96-8baa-42d9-834b-825cc9d31b91 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:36.016016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1246/2470] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:36.668993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e37e2-a3e2-4dc5-a932-3492d2d2694b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:36.670721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-be6e37e2-a3e2-4dc5-a932-3492d2d2694b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:36.684795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e37e2-a3e2-4dc5-a932-3492d2d2694b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:36.689418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42be852-8dd8-4b8c-9416-9b2fa2b3b333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:36.690181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42be852-8dd8-4b8c-9416-9b2fa2b3b333 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:36.690506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42be852-8dd8-4b8c-9416-9b2fa2b3b333 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:36.690890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42be852-8dd8-4b8c-9416-9b2fa2b3b333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:36.763985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42be852-8dd8-4b8c-9416-9b2fa2b3b333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:36.766678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42be852-8dd8-4b8c-9416-9b2fa2b3b333 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:36.778499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1225/2471] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:40.853859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b80a96-8baa-42d9-834b-825cc9d31b91 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:40.862697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b80a96-8baa-42d9-834b-825cc9d31b91 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:40.866576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3908e-b5c8-4219-bff9-67d92aeff69d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:40.867382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3908e-b5c8-4219-bff9-67d92aeff69d service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:40.867793 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3908e-b5c8-4219-bff9-67d92aeff69d service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:40.868125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3908e-b5c8-4219-bff9-67d92aeff69d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:40.939634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3908e-b5c8-4219-bff9-67d92aeff69d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:40.941087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3908e-b5c8-4219-bff9-67d92aeff69d service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:40.952369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1247/2472] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:40.961803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42be852-8dd8-4b8c-9416-9b2fa2b3b333 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:40.963809 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b42be852-8dd8-4b8c-9416-9b2fa2b3b333 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:41.004315 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:41.034164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42be852-8dd8-4b8c-9416-9b2fa2b3b333 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:41.039822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84042128-76f1-4a74-a1b3-0dcecf1eefd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:41.041004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84042128-76f1-4a74-a1b3-0dcecf1eefd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:41.041401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84042128-76f1-4a74-a1b3-0dcecf1eefd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:41.041794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84042128-76f1-4a74-a1b3-0dcecf1eefd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:41.044030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-84042128-76f1-4a74-a1b3-0dcecf1eefd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:41.127004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84042128-76f1-4a74-a1b3-0dcecf1eefd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:41.128517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84042128-76f1-4a74-a1b3-0dcecf1eefd4 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:41.141087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1226/2473] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:43.268750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3908e-b5c8-4219-bff9-67d92aeff69d service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:43.289978 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84042128-76f1-4a74-a1b3-0dcecf1eefd4 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:43.302468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84042128-76f1-4a74-a1b3-0dcecf1eefd4 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:43.304465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3908e-b5c8-4219-bff9-67d92aeff69d service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:43.314814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a904d7b-ed63-451e-b679-1a9735c89785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:43.315004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd95e79-bedf-44d7-9de9-d357f093d4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:43.316034 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd95e79-bedf-44d7-9de9-d357f093d4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:43.317140 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a904d7b-ed63-451e-b679-1a9735c89785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:43.317531 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd95e79-bedf-44d7-9de9-d357f093d4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:43.317926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a904d7b-ed63-451e-b679-1a9735c89785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:43.318299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd95e79-bedf-44d7-9de9-d357f093d4b6 admin admin] PATH_INFO: `/v3/users/1ad2312ac7694e608b9557a12d73e30c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:43.318663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a904d7b-ed63-451e-b679-1a9735c89785 admin admin] PATH_INFO: `/v3/users/5bc4fde81b7645b6aa0b5c1f2c28d313` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:43.320475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd95e79-bedf-44d7-9de9-d357f093d4b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ad2312ac7694e608b9557a12d73e30c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:43.321361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a904d7b-ed63-451e-b679-1a9735c89785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc4fde81b7645b6aa0b5c1f2c28d313)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:43.323634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd95e79-bedf-44d7-9de9-d357f093d4b6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:43.325661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a904d7b-ed63-451e-b679-1a9735c89785 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:43.334644 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:43.420953 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ffd95e79-bedf-44d7-9de9-d357f093d4b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ad2312ac7694e608b9557a12d73e30c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:43.421997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ffd95e79-bedf-44d7-9de9-d357f093d4b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ad2312ac7694e608b9557a12d73e30c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:43.422640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ffd95e79-bedf-44d7-9de9-d357f093d4b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ad2312ac7694e608b9557a12d73e30c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:43.430358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0a904d7b-ed63-451e-b679-1a9735c89785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc4fde81b7645b6aa0b5c1f2c28d313'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:43.438718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ffd95e79-bedf-44d7-9de9-d357f093d4b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ad2312ac7694e608b9557a12d73e30c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:43.443741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0a904d7b-ed63-451e-b679-1a9735c89785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc4fde81b7645b6aa0b5c1f2c28d313'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:43.445913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0a904d7b-ed63-451e-b679-1a9735c89785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc4fde81b7645b6aa0b5c1f2c28d313'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:43.452234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ffd95e79-bedf-44d7-9de9-d357f093d4b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ad2312ac7694e608b9557a12d73e30c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:43.478952 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0a904d7b-ed63-451e-b679-1a9735c89785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc4fde81b7645b6aa0b5c1f2c28d313'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:43.482158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1248/2474] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:43 2022] DELETE /identity/v3/users/1ad2312ac7694e608b9557a12d73e30c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:43.488298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788aa9-a808-4fe2-85a8-c172332bee4e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:43.488734 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788aa9-a808-4fe2-85a8-c172332bee4e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:43.488867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788aa9-a808-4fe2-85a8-c172332bee4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:43.489852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0a904d7b-ed63-451e-b679-1a9735c89785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc4fde81b7645b6aa0b5c1f2c28d313'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:43.498668 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1227/2475] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:43 2022] DELETE /identity/v3/users/5bc4fde81b7645b6aa0b5c1f2c28d313 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:43.503072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35e178-b429-407e-80c5-c4cc69cf586b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:43.503538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35e178-b429-407e-80c5-c4cc69cf586b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:43.503821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35e178-b429-407e-80c5-c4cc69cf586b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:43.509330 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f1788aa9-a808-4fe2-85a8-c172332bee4e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:43.511472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f1788aa9-a808-4fe2-85a8-c172332bee4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:43.522224 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cf35e178-b429-407e-80c5-c4cc69cf586b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:43.523460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf35e178-b429-407e-80c5-c4cc69cf586b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:43.578191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:43.584409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1249/2476] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:27:43.592886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1788aa9-a808-4fe2-85a8-c172332bee4e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:43.595319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f1788aa9-a808-4fe2-85a8-c172332bee4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:43.596618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1228/2477] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:27:43.603495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35e178-b429-407e-80c5-c4cc69cf586b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:43.606267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf35e178-b429-407e-80c5-c4cc69cf586b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:43.668482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1788aa9-a808-4fe2-85a8-c172332bee4e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:43.672475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4db646-09ec-4f28-8e28-3fb95b538a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:43.673281 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db646-09ec-4f28-8e28-3fb95b538a56 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:43.673576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db646-09ec-4f28-8e28-3fb95b538a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:43.673892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db646-09ec-4f28-8e28-3fb95b538a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:43.677457 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35e178-b429-407e-80c5-c4cc69cf586b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:43.681538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d51a3-0175-42c0-897a-f3384e3af880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:43.682192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d51a3-0175-42c0-897a-f3384e3af880 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:43.682507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d51a3-0175-42c0-897a-f3384e3af880 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:43.683536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d51a3-0175-42c0-897a-f3384e3af880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:43.684577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4db646-09ec-4f28-8e28-3fb95b538a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:43.686212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4db646-09ec-4f28-8e28-3fb95b538a56 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:43.694990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d51a3-0175-42c0-897a-f3384e3af880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:43.696628 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d51a3-0175-42c0-897a-f3384e3af880 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:43.698568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1250/2478] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:27:43.708050 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1229/2479] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:27:44.211744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4db646-09ec-4f28-8e28-3fb95b538a56 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:44.213197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ae4db646-09ec-4f28-8e28-3fb95b538a56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:44.226051 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d51a3-0175-42c0-897a-f3384e3af880 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:44.226419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4db646-09ec-4f28-8e28-3fb95b538a56 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:44.228067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5b5d51a3-0175-42c0-897a-f3384e3af880 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:44.249067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:44.250110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:44.250415 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:44.250953 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] PATH_INFO: `/v3/projects/e7f882a220d24bd5b295deceac26280e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:44.252364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7f882a220d24bd5b295deceac26280e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:44.256359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d51a3-0175-42c0-897a-f3384e3af880 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:44.260094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:44.272522 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:44.273556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:44.273785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:44.283469 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] PATH_INFO: `/v3/projects/275be89764304e3aaea606fd8db79af1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:44.285973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] RBAC: Authorizing `identity:delete_project(project_id=275be89764304e3aaea606fd8db79af1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:44.288369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:44.307372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:44.353194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7f882a220d24bd5b295deceac26280e', 'request_id': 'req-0868f5f0-953b-4689-9fb2-342444251b8b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.361848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7f882a220d24bd5b295deceac26280e', 'request_id': 'req-0868f5f0-953b-4689-9fb2-342444251b8b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.372852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7f882a220d24bd5b295deceac26280e', 'request_id': 'req-0868f5f0-953b-4689-9fb2-342444251b8b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.373825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '275be89764304e3aaea606fd8db79af1', 'request_id': 'req-cdda4063-e44f-46ba-b29f-af19cdc4c94e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.381557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '275be89764304e3aaea606fd8db79af1', 'request_id': 'req-cdda4063-e44f-46ba-b29f-af19cdc4c94e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.388974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] The token cache is being invalidate because project e7f882a220d24bd5b295deceac26280e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:44.389752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.389972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '275be89764304e3aaea606fd8db79af1', 'request_id': 'req-cdda4063-e44f-46ba-b29f-af19cdc4c94e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.390775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.398490 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1251/2480] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:44 2022] DELETE /identity/v3/projects/e7f882a220d24bd5b295deceac26280e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:44.399677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:44.402239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:44.413100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] The token cache is being invalidate because project 275be89764304e3aaea606fd8db79af1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:44.413390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.414579 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cdda4063-e44f-46ba-b29f-af19cdc4c94e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.416585 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1230/2481] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:44 2022] DELETE /identity/v3/projects/275be89764304e3aaea606fd8db79af1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:44.422087 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb7d48-29e7-4982-9674-2982e01d3a6f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:44.422672 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb7d48-29e7-4982-9674-2982e01d3a6f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:44.423200 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb7d48-29e7-4982-9674-2982e01d3a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:44.459003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d8bb7d48-29e7-4982-9674-2982e01d3a6f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:44.461306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d8bb7d48-29e7-4982-9674-2982e01d3a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:44.491169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868f5f0-953b-4689-9fb2-342444251b8b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:44.583531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1231/2482] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:27:44.585852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31348cb-8bee-4fa1-b372-6d7f2fd2fa1b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:44.585852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31348cb-8bee-4fa1-b372-6d7f2fd2fa1b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:44.586146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31348cb-8bee-4fa1-b372-6d7f2fd2fa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:44.591028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:44.592115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:44.592468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:44.592980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] PATH_INFO: `/v3/users/399047e53fd84357a523409113d8e701` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:44.594611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=399047e53fd84357a523409113d8e701)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:44.596966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:44.608178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e31348cb-8bee-4fa1-b372-6d7f2fd2fa1b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:44.609479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e31348cb-8bee-4fa1-b372-6d7f2fd2fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:44.621600 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1232/2483] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:44.626123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31348cb-8bee-4fa1-b372-6d7f2fd2fa1b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:44.636362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31348cb-8bee-4fa1-b372-6d7f2fd2fa1b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:44.642678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa2d03-7b75-4f50-9701-66fccb7a5258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:44.643864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa2d03-7b75-4f50-9701-66fccb7a5258 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:44.644387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa2d03-7b75-4f50-9701-66fccb7a5258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:44.644684 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa2d03-7b75-4f50-9701-66fccb7a5258 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:44.645608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa2d03-7b75-4f50-9701-66fccb7a5258 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:27:44.646067 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa2d03-7b75-4f50-9701-66fccb7a5258 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:44.647702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa2d03-7b75-4f50-9701-66fccb7a5258 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:44.670975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1233/2484] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:27:44.671995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa2d03-7b75-4f50-9701-66fccb7a5258 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:44.673842 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ceaa2d03-7b75-4f50-9701-66fccb7a5258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:44.681948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:44.688189 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa2d03-7b75-4f50-9701-66fccb7a5258 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:44.691746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '399047e53fd84357a523409113d8e701'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.692572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '399047e53fd84357a523409113d8e701'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.693363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '399047e53fd84357a523409113d8e701'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.696025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:44.696886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:44.697802 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:44.698169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] PATH_INFO: `/v3/users/9178e89a790d48d19b25c4c22bd0ca4e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:44.699890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9178e89a790d48d19b25c4c22bd0ca4e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:44.702707 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:44.706434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:44.715725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '399047e53fd84357a523409113d8e701'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.729655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '399047e53fd84357a523409113d8e701'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.746778 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1252/2485] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:44 2022] DELETE /identity/v3/users/399047e53fd84357a523409113d8e701 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:44.750101 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:44.750871 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:44.769813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9178e89a790d48d19b25c4c22bd0ca4e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.778169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9178e89a790d48d19b25c4c22bd0ca4e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.778894 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9178e89a790d48d19b25c4c22bd0ca4e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.796095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9178e89a790d48d19b25c4c22bd0ca4e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.803664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9178e89a790d48d19b25c4c22bd0ca4e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:44.812299 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1234/2486] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:44 2022] DELETE /identity/v3/users/9178e89a790d48d19b25c4c22bd0ca4e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:44.834144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573b284-fb2f-4853-a524-64ec9aaa8b74 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:44.900003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee81b4-87b7-46e9-82d9-543c1fb6dd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:44.900670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee81b4-87b7-46e9-82d9-543c1fb6dd4c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:44.900952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee81b4-87b7-46e9-82d9-543c1fb6dd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:44.901320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee81b4-87b7-46e9-82d9-543c1fb6dd4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:44.902613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee81b4-87b7-46e9-82d9-543c1fb6dd4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:44.905128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee81b4-87b7-46e9-82d9-543c1fb6dd4c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:44.931275 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1253/2487] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:44 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:44.937025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:44.938614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:44.952592 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d1a3e-b5d1-42c7-bc8b-f540e93b49c9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.030493 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021b309-db49-45c8-8ea0-a28021242ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.031733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021b309-db49-45c8-8ea0-a28021242ecb admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.031985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021b309-db49-45c8-8ea0-a28021242ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.032461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021b309-db49-45c8-8ea0-a28021242ecb admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.033622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021b309-db49-45c8-8ea0-a28021242ecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.036483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021b309-db49-45c8-8ea0-a28021242ecb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.063346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1235/2488] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:44 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:27:45.067698 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee81b4-87b7-46e9-82d9-543c1fb6dd4c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.074006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021b309-db49-45c8-8ea0-a28021242ecb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.075854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a021b309-db49-45c8-8ea0-a28021242ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:45.079651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee81b4-87b7-46e9-82d9-543c1fb6dd4c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.088216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021b309-db49-45c8-8ea0-a28021242ecb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.095138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.095955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.096236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.096509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] PATH_INFO: `/v3/projects/e916473814d44ac1ae6339f0d052daa1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.098035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] RBAC: Authorizing `identity:delete_project(project_id=e916473814d44ac1ae6339f0d052daa1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.099955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.112677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6de4-b16d-4794-8b54-17edd5c378e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.113493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6de4-b16d-4794-8b54-17edd5c378e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.114695 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6de4-b16d-4794-8b54-17edd5c378e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.114986 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6de4-b16d-4794-8b54-17edd5c378e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.115812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc6de4-b16d-4794-8b54-17edd5c378e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.117624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc6de4-b16d-4794-8b54-17edd5c378e0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.132054 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1254/2489] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:45.133601 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6de4-b16d-4794-8b54-17edd5c378e0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.136884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e7bc6de4-b16d-4794-8b54-17edd5c378e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:45.150806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6de4-b16d-4794-8b54-17edd5c378e0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.163789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e916473814d44ac1ae6339f0d052daa1', 'request_id': 'req-1a03e884-51d2-45de-b470-49203c702cda'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.170687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e916473814d44ac1ae6339f0d052daa1', 'request_id': 'req-1a03e884-51d2-45de-b470-49203c702cda'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.171819 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=80,keystone:DELETE=16,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:45.179814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e916473814d44ac1ae6339f0d052daa1', 'request_id': 'req-1a03e884-51d2-45de-b470-49203c702cda'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.194018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] The token cache is being invalidate because project e916473814d44ac1ae6339f0d052daa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:45.194309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.194957 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.196518 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1236/2490] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:45 2022] DELETE /identity/v3/projects/e916473814d44ac1ae6339f0d052daa1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:45.200599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.200599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:45.217028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:45.225325 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.226386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.227530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.227846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] PATH_INFO: `/v3/projects/b624e96fb3354bdc8220c2b367ab4946` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.229151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b624e96fb3354bdc8220c2b367ab4946)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.231891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.270230 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03e884-51d2-45de-b470-49203c702cda admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.283202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b624e96fb3354bdc8220c2b367ab4946', 'request_id': 'req-0070f5a3-b55a-450a-95fb-1b455142891b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.289972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b624e96fb3354bdc8220c2b367ab4946', 'request_id': 'req-0070f5a3-b55a-450a-95fb-1b455142891b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.298803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b624e96fb3354bdc8220c2b367ab4946', 'request_id': 'req-0070f5a3-b55a-450a-95fb-1b455142891b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.311623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] The token cache is being invalidate because project b624e96fb3354bdc8220c2b367ab4946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:45.312260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.312795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0070f5a3-b55a-450a-95fb-1b455142891b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.314431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1255/2491] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:45 2022] DELETE /identity/v3/projects/b624e96fb3354bdc8220c2b367ab4946 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:45.316210 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b85c8-c051-4fa4-a8d4-7ee202999a1e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.316491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b85c8-c051-4fa4-a8d4-7ee202999a1e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.316770 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b85c8-c051-4fa4-a8d4-7ee202999a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.329310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef1184-4728-487d-afe7-2378ac2ebc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.330096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef1184-4728-487d-afe7-2378ac2ebc69 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.330376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef1184-4728-487d-afe7-2378ac2ebc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.330654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef1184-4728-487d-afe7-2378ac2ebc69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.331405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef1184-4728-487d-afe7-2378ac2ebc69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.332988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef1184-4728-487d-afe7-2378ac2ebc69 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.335490 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b80b85c8-c051-4fa4-a8d4-7ee202999a1e None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:45.336693 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b80b85c8-c051-4fa4-a8d4-7ee202999a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:45.346778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1237/2492] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:45.346778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef1184-4728-487d-afe7-2378ac2ebc69 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.348011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1256/2493] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:45.348397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-84ef1184-4728-487d-afe7-2378ac2ebc69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:45.348885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b85c8-c051-4fa4-a8d4-7ee202999a1e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.357016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b80b85c8-c051-4fa4-a8d4-7ee202999a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:45.368290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef1184-4728-487d-afe7-2378ac2ebc69 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.371141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b85c8-c051-4fa4-a8d4-7ee202999a1e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.379345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a21e4d-197b-4f3a-a91e-defd535421bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.379527 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf0fb2-bcef-4d66-a577-4324cac00507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.380253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf0fb2-bcef-4d66-a577-4324cac00507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.380389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a21e4d-197b-4f3a-a91e-defd535421bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.380535 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf0fb2-bcef-4d66-a577-4324cac00507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.380652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a21e4d-197b-4f3a-a91e-defd535421bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.380806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf0fb2-bcef-4d66-a577-4324cac00507 admin admin] PATH_INFO: `/v3/users/8be5803e7a1d4114a58f578a1ae1f500` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.380918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a21e4d-197b-4f3a-a91e-defd535421bd admin admin] PATH_INFO: `/v3/projects/8f1295403805438d82a585fb1abd9c2a/users/5ae4839689d14a95bfe7404d62cb8d08/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.382242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf0fb2-bcef-4d66-a577-4324cac00507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be5803e7a1d4114a58f578a1ae1f500)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.384511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf0fb2-bcef-4d66-a577-4324cac00507 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.396356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a21e4d-197b-4f3a-a91e-defd535421bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f1295403805438d82a585fb1abd9c2a, user_id=5ae4839689d14a95bfe7404d62cb8d08, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.399197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a21e4d-197b-4f3a-a91e-defd535421bd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.415610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1257/2494] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:45 2022] PUT /identity/v3/projects/8f1295403805438d82a585fb1abd9c2a/users/5ae4839689d14a95bfe7404d62cb8d08/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:45.418263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e604c8-c5b4-4a5c-a5f8-ab206f9549af None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.418615 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e604c8-c5b4-4a5c-a5f8-ab206f9549af None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.418959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e604c8-c5b4-4a5c-a5f8-ab206f9549af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.442809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-87e604c8-c5b4-4a5c-a5f8-ab206f9549af None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:45.444082 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-87e604c8-c5b4-4a5c-a5f8-ab206f9549af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:45.457381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-05bf0fb2-bcef-4d66-a577-4324cac00507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be5803e7a1d4114a58f578a1ae1f500'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.467951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-05bf0fb2-bcef-4d66-a577-4324cac00507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be5803e7a1d4114a58f578a1ae1f500'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.468729 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-05bf0fb2-bcef-4d66-a577-4324cac00507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be5803e7a1d4114a58f578a1ae1f500'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.484041 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-05bf0fb2-bcef-4d66-a577-4324cac00507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be5803e7a1d4114a58f578a1ae1f500'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.491578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-05bf0fb2-bcef-4d66-a577-4324cac00507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be5803e7a1d4114a58f578a1ae1f500'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.501089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1238/2495] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:45 2022] DELETE /identity/v3/users/8be5803e7a1d4114a58f578a1ae1f500 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:45.501089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ade31-4f5b-44e6-8bdb-bf7e2d0cd444 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.501089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ade31-4f5b-44e6-8bdb-bf7e2d0cd444 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.501089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ade31-4f5b-44e6-8bdb-bf7e2d0cd444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.512788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:45.520360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1258/2496] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:45.527264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e604c8-c5b4-4a5c-a5f8-ab206f9549af None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.538062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e604c8-c5b4-4a5c-a5f8-ab206f9549af None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.540988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-927ade31-4f5b-44e6-8bdb-bf7e2d0cd444 None None] MFA Rules not processed for user `5ae4839689d14a95bfe7404d62cb8d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:45.543104 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-927ade31-4f5b-44e6-8bdb-bf7e2d0cd444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:45.609048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:45.616887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645838ac-e300-46d9-b6fd-c627595a4f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.617969 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645838ac-e300-46d9-b6fd-c627595a4f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.618366 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645838ac-e300-46d9-b6fd-c627595a4f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.618985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645838ac-e300-46d9-b6fd-c627595a4f24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.620752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645838ac-e300-46d9-b6fd-c627595a4f24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:27:45.621308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645838ac-e300-46d9-b6fd-c627595a4f24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.624688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645838ac-e300-46d9-b6fd-c627595a4f24 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.627236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1239/2497] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:45.642459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ade31-4f5b-44e6-8bdb-bf7e2d0cd444 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.646589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1259/2498] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:27:45.647589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-927ade31-4f5b-44e6-8bdb-bf7e2d0cd444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:45.647731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645838ac-e300-46d9-b6fd-c627595a4f24 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.660735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645838ac-e300-46d9-b6fd-c627595a4f24 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.679560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f008e-d81c-436e-9556-120e40551f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.680245 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f008e-d81c-436e-9556-120e40551f65 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.680532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f008e-d81c-436e-9556-120e40551f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.680812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f008e-d81c-436e-9556-120e40551f65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.681529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f008e-d81c-436e-9556-120e40551f65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.686001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f008e-d81c-436e-9556-120e40551f65 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.722794 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1260/2499] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:45 2022] POST /identity/v3/projects => generated 355 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:27:45.724282 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f008e-d81c-436e-9556-120e40551f65 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.750504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ade31-4f5b-44e6-8bdb-bf7e2d0cd444 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.794632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f008e-d81c-436e-9556-120e40551f65 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.802337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861ac14-007f-4f48-84a8-fe4380581c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.803030 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861ac14-007f-4f48-84a8-fe4380581c2b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.803355 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861ac14-007f-4f48-84a8-fe4380581c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.804153 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861ac14-007f-4f48-84a8-fe4380581c2b admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.804872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861ac14-007f-4f48-84a8-fe4380581c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.808145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861ac14-007f-4f48-84a8-fe4380581c2b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.835079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc1efd-ae38-45e4-a383-01105c1600ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.835731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc1efd-ae38-45e4-a383-01105c1600ac service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.836021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc1efd-ae38-45e4-a383-01105c1600ac service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.836301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc1efd-ae38-45e4-a383-01105c1600ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.836710 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1261/2500] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:45 2022] POST /identity/v3/users => generated 278 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:45.837491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861ac14-007f-4f48-84a8-fe4380581c2b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.839148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b861ac14-007f-4f48-84a8-fe4380581c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:45.850979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861ac14-007f-4f48-84a8-fe4380581c2b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.856207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:45.857057 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.857344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.857621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] PATH_INFO: `/v3/projects/5665680c7ec74be9b58a42a3f14db531` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.858848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5665680c7ec74be9b58a42a3f14db531)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.860767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.867288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:45.899601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc1efd-ae38-45e4-a383-01105c1600ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:45.901303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc1efd-ae38-45e4-a383-01105c1600ac service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:45.908148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5665680c7ec74be9b58a42a3f14db531', 'request_id': 'req-0baf7117-8d95-426c-a1bb-f84b50c9581e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.910731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1240/2501] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:27:45.911818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc1efd-ae38-45e4-a383-01105c1600ac service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:45.916017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5665680c7ec74be9b58a42a3f14db531', 'request_id': 'req-0baf7117-8d95-426c-a1bb-f84b50c9581e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.922879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc1efd-ae38-45e4-a383-01105c1600ac service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:45.926936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5665680c7ec74be9b58a42a3f14db531', 'request_id': 'req-0baf7117-8d95-426c-a1bb-f84b50c9581e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.943293 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] The token cache is being invalidate because project 5665680c7ec74be9b58a42a3f14db531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:45.943293 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.943636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0baf7117-8d95-426c-a1bb-f84b50c9581e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:45.945203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1262/2502] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:45 2022] DELETE /identity/v3/projects/5665680c7ec74be9b58a42a3f14db531 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:45.952743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db64e9-5b4a-4bcc-99f9-02dc765e319b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:45.953038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db64e9-5b4a-4bcc-99f9-02dc765e319b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:45.953322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db64e9-5b4a-4bcc-99f9-02dc765e319b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:45.980165 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-19db64e9-5b4a-4bcc-99f9-02dc765e319b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:45.981368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-19db64e9-5b4a-4bcc-99f9-02dc765e319b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.005508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03d25a-2dee-49cf-a694-bf438ef00673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.006240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03d25a-2dee-49cf-a694-bf438ef00673 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.006528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03d25a-2dee-49cf-a694-bf438ef00673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.006803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03d25a-2dee-49cf-a694-bf438ef00673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.007525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03d25a-2dee-49cf-a694-bf438ef00673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.009225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03d25a-2dee-49cf-a694-bf438ef00673 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.026632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1241/2503] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:46.029696 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:46.030205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03d25a-2dee-49cf-a694-bf438ef00673 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.032113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1d03d25a-2dee-49cf-a694-bf438ef00673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.046869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03d25a-2dee-49cf-a694-bf438ef00673 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.052841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e04f7-d0c7-4a80-bb5c-73d93741c313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.053524 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e04f7-d0c7-4a80-bb5c-73d93741c313 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.053845 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e04f7-d0c7-4a80-bb5c-73d93741c313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.054129 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e04f7-d0c7-4a80-bb5c-73d93741c313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.054857 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e04f7-d0c7-4a80-bb5c-73d93741c313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.056458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e04f7-d0c7-4a80-bb5c-73d93741c313 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.065369 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1263/2504] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:46.067412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3dd4-8ba1-4967-85b7-f2061e081cfe None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.068006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3dd4-8ba1-4967-85b7-f2061e081cfe None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.068006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3dd4-8ba1-4967-85b7-f2061e081cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.068202 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1242/2505] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:46.071660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e04f7-d0c7-4a80-bb5c-73d93741c313 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.083898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e04f7-d0c7-4a80-bb5c-73d93741c313 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.088314 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7e8b3dd4-8ba1-4967-85b7-f2061e081cfe None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:46.089018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ebe8e-d81d-4c48-af61-2a1059191fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.089916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7e8b3dd4-8ba1-4967-85b7-f2061e081cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.090255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ebe8e-d81d-4c48-af61-2a1059191fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.090588 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ebe8e-d81d-4c48-af61-2a1059191fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.090889 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ebe8e-d81d-4c48-af61-2a1059191fbc admin admin] PATH_INFO: `/v3/projects/b040cd35a96f4f91be5c2a9d8bf9e4d7/users/1a0e00f42d39426184c802d087d6c531/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.105365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1264/2506] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:46.105365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b3dd4-8ba1-4967-85b7-f2061e081cfe None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.113766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ebe8e-d81d-4c48-af61-2a1059191fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b040cd35a96f4f91be5c2a9d8bf9e4d7, user_id=1a0e00f42d39426184c802d087d6c531, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.114342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ebe8e-d81d-4c48-af61-2a1059191fbc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.115137 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b3dd4-8ba1-4967-85b7-f2061e081cfe None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.119856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578296c4-838f-4f05-90b2-71f72eab0334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.121535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578296c4-838f-4f05-90b2-71f72eab0334 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.121873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578296c4-838f-4f05-90b2-71f72eab0334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.122149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578296c4-838f-4f05-90b2-71f72eab0334 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.122882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578296c4-838f-4f05-90b2-71f72eab0334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:27:46.123186 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578296c4-838f-4f05-90b2-71f72eab0334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.124773 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578296c4-838f-4f05-90b2-71f72eab0334 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.132500 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1243/2507] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:46 2022] PUT /identity/v3/projects/b040cd35a96f4f91be5c2a9d8bf9e4d7/users/1a0e00f42d39426184c802d087d6c531/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:27:46.138768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1265/2508] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:27:46.146725 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ebe8e-d81d-4c48-af61-2a1059191fbc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.147202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ae4d6-e1d8-4c16-a9fc-9bd0ff1bc0ab None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.147526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ae4d6-e1d8-4c16-a9fc-9bd0ff1bc0ab None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.147803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ae4d6-e1d8-4c16-a9fc-9bd0ff1bc0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.177339 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-188ae4d6-e1d8-4c16-a9fc-9bd0ff1bc0ab None None] MFA Rules not processed for user `1a0e00f42d39426184c802d087d6c531`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:46.178589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-188ae4d6-e1d8-4c16-a9fc-9bd0ff1bc0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.217614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ebe8e-d81d-4c48-af61-2a1059191fbc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.222277 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791efcf8-6180-4db3-a15d-31bd6cd3fbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.222930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791efcf8-6180-4db3-a15d-31bd6cd3fbb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.223212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791efcf8-6180-4db3-a15d-31bd6cd3fbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.223589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791efcf8-6180-4db3-a15d-31bd6cd3fbb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.224154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791efcf8-6180-4db3-a15d-31bd6cd3fbb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.226076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791efcf8-6180-4db3-a15d-31bd6cd3fbb9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.244607 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1266/2509] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:46 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:46.246203 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ae4d6-e1d8-4c16-a9fc-9bd0ff1bc0ab None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.247474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-188ae4d6-e1d8-4c16-a9fc-9bd0ff1bc0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.255636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1244/2510] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:46 2022] POST /identity/v3/projects => generated 401 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:46.257600 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791efcf8-6180-4db3-a15d-31bd6cd3fbb9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.261652 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ae4d6-e1d8-4c16-a9fc-9bd0ff1bc0ab None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.274575 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:46.320757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.321639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.321947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.322221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] PATH_INFO: `/v3/users/5ae4839689d14a95bfe7404d62cb8d08` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.324335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae4839689d14a95bfe7404d62cb8d08)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.326135 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791efcf8-6180-4db3-a15d-31bd6cd3fbb9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.326345 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.330948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bedeaa-6fd9-45fc-b20b-d4620ca022ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.331608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bedeaa-6fd9-45fc-b20b-d4620ca022ac admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.331916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bedeaa-6fd9-45fc-b20b-d4620ca022ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.332198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bedeaa-6fd9-45fc-b20b-d4620ca022ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.332877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bedeaa-6fd9-45fc-b20b-d4620ca022ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.334826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bedeaa-6fd9-45fc-b20b-d4620ca022ac admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.362506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1245/2511] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:46 2022] POST /identity/v3/projects => generated 357 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:46.363311 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bedeaa-6fd9-45fc-b20b-d4620ca022ac admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.378503 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae4839689d14a95bfe7404d62cb8d08'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:46.379178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae4839689d14a95bfe7404d62cb8d08'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:46.379798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae4839689d14a95bfe7404d62cb8d08'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:46.384908 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:46.394251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae4839689d14a95bfe7404d62cb8d08'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:46.400893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae4839689d14a95bfe7404d62cb8d08'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:46.414712 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1267/2512] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:46 2022] DELETE /identity/v3/users/5ae4839689d14a95bfe7404d62cb8d08 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:46.415492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.416836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.429429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bedeaa-6fd9-45fc-b20b-d4620ca022ac admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.486904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6a2f9-7dbf-4d7c-9418-7ebad4ef2c6f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.491818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891087e-c291-4368-a929-a23c7d6f99b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.492492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891087e-c291-4368-a929-a23c7d6f99b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.492783 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891087e-c291-4368-a929-a23c7d6f99b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.493069 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891087e-c291-4368-a929-a23c7d6f99b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.493784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891087e-c291-4368-a929-a23c7d6f99b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.495925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c816d-e1d3-44e1-8656-52c34fbb5d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.496476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891087e-c291-4368-a929-a23c7d6f99b0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.496779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c816d-e1d3-44e1-8656-52c34fbb5d7b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.497062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c816d-e1d3-44e1-8656-52c34fbb5d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.497334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c816d-e1d3-44e1-8656-52c34fbb5d7b admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.498071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c816d-e1d3-44e1-8656-52c34fbb5d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.499973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c816d-e1d3-44e1-8656-52c34fbb5d7b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.518049 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1268/2513] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:46 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:46.519319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38844f7d-989b-4005-8d2f-0b43f538bdb5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.519611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38844f7d-989b-4005-8d2f-0b43f538bdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.519889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38844f7d-989b-4005-8d2f-0b43f538bdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.523406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1246/2514] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:46 2022] POST /identity/v3/users => generated 301 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:46.524562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c816d-e1d3-44e1-8656-52c34fbb5d7b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.535892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c816d-e1d3-44e1-8656-52c34fbb5d7b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.537576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-38844f7d-989b-4005-8d2f-0b43f538bdb5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:46.538866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-38844f7d-989b-4005-8d2f-0b43f538bdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.540848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bd51f-f1ad-4a94-ae65-96c893c0c138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.541495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bd51f-f1ad-4a94-ae65-96c893c0c138 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.542431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bd51f-f1ad-4a94-ae65-96c893c0c138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.542770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bd51f-f1ad-4a94-ae65-96c893c0c138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.543489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0bd51f-f1ad-4a94-ae65-96c893c0c138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.545206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0bd51f-f1ad-4a94-ae65-96c893c0c138 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.547081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1269/2515] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:46.547864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38844f7d-989b-4005-8d2f-0b43f538bdb5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.549119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-38844f7d-989b-4005-8d2f-0b43f538bdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.561398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38844f7d-989b-4005-8d2f-0b43f538bdb5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.566562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1247/2516] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:46.566562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bd51f-f1ad-4a94-ae65-96c893c0c138 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.568324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0e0bd51f-f1ad-4a94-ae65-96c893c0c138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.569756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839761a1-1297-4a7b-b5da-7efd0937e008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.570431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839761a1-1297-4a7b-b5da-7efd0937e008 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.570765 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839761a1-1297-4a7b-b5da-7efd0937e008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.570998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839761a1-1297-4a7b-b5da-7efd0937e008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.571716 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839761a1-1297-4a7b-b5da-7efd0937e008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.573583 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839761a1-1297-4a7b-b5da-7efd0937e008 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.584326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1270/2517] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:46.585117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839761a1-1297-4a7b-b5da-7efd0937e008 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.595509 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839761a1-1297-4a7b-b5da-7efd0937e008 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.600115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d129875-8d25-4cac-9f8c-5458ae7c4ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.600981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d129875-8d25-4cac-9f8c-5458ae7c4ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.601125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:46.601271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d129875-8d25-4cac-9f8c-5458ae7c4ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.601551 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d129875-8d25-4cac-9f8c-5458ae7c4ff1 admin admin] PATH_INFO: `/v3/projects/c166cdab2dab4e0ab3a5d090ebfaa772/users/7bfd21ee1c504996943465b7158e6b19/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.617288 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d129875-8d25-4cac-9f8c-5458ae7c4ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c166cdab2dab4e0ab3a5d090ebfaa772, user_id=7bfd21ee1c504996943465b7158e6b19, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.621011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d129875-8d25-4cac-9f8c-5458ae7c4ff1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.633851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bd51f-f1ad-4a94-ae65-96c893c0c138 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.634530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1271/2518] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:46 2022] PUT /identity/v3/projects/c166cdab2dab4e0ab3a5d090ebfaa772/users/7bfd21ee1c504996943465b7158e6b19/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:46.635414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d129875-8d25-4cac-9f8c-5458ae7c4ff1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.684692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba7c2f-7e82-448e-a76e-5f23b29642cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.685346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba7c2f-7e82-448e-a76e-5f23b29642cf service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.685632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba7c2f-7e82-448e-a76e-5f23b29642cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.685953 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba7c2f-7e82-448e-a76e-5f23b29642cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.702742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d129875-8d25-4cac-9f8c-5458ae7c4ff1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.707598 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ad225-f4e9-40b8-9182-df7c32ec78fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.708266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ad225-f4e9-40b8-9182-df7c32ec78fe admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.708553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ad225-f4e9-40b8-9182-df7c32ec78fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.708831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ad225-f4e9-40b8-9182-df7c32ec78fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.709589 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ad225-f4e9-40b8-9182-df7c32ec78fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.711222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ad225-f4e9-40b8-9182-df7c32ec78fe admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.721283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1272/2519] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:46.725179 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ad225-f4e9-40b8-9182-df7c32ec78fe admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.735407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ad225-f4e9-40b8-9182-df7c32ec78fe admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.740097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35787b-722f-4303-a3ef-866032ecea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.740757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35787b-722f-4303-a3ef-866032ecea2f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.741045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35787b-722f-4303-a3ef-866032ecea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.741337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35787b-722f-4303-a3ef-866032ecea2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.742082 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35787b-722f-4303-a3ef-866032ecea2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.743654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35787b-722f-4303-a3ef-866032ecea2f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.749704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba7c2f-7e82-448e-a76e-5f23b29642cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.751134 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba7c2f-7e82-448e-a76e-5f23b29642cf service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.753367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1273/2520] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:46.754167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35787b-722f-4303-a3ef-866032ecea2f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.760258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:46.763169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1248/2521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:46.765555 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35787b-722f-4303-a3ef-866032ecea2f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.771887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba7c2f-7e82-448e-a76e-5f23b29642cf service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.773769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b9d5e-d1b1-48be-8db1-606b18facb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.775119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b9d5e-d1b1-48be-8db1-606b18facb5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.775354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b9d5e-d1b1-48be-8db1-606b18facb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.775563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b9d5e-d1b1-48be-8db1-606b18facb5d admin admin] PATH_INFO: `/v3/projects/3c12fc2d8be14b32ae64f1f8797e1311/users/1b2c6a31169f404699d9ab5309b71c72/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.789013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba7c2f-7e82-448e-a76e-5f23b29642cf service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:46.794149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b9d5e-d1b1-48be-8db1-606b18facb5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c12fc2d8be14b32ae64f1f8797e1311, user_id=1b2c6a31169f404699d9ab5309b71c72, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.795044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c45cd-1258-4387-ac64-e917b5b99b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:46.795941 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c45cd-1258-4387-ac64-e917b5b99b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.796225 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c45cd-1258-4387-ac64-e917b5b99b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.796506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c45cd-1258-4387-ac64-e917b5b99b46 admin admin] PATH_INFO: `/v3/projects/c166cdab2dab4e0ab3a5d090ebfaa772/users/7bfd21ee1c504996943465b7158e6b19/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.796811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b9d5e-d1b1-48be-8db1-606b18facb5d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.798515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c45cd-1258-4387-ac64-e917b5b99b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c166cdab2dab4e0ab3a5d090ebfaa772, user_id=7bfd21ee1c504996943465b7158e6b19, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:46.801464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c45cd-1258-4387-ac64-e917b5b99b46 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:46.813316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1249/2522] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:46 2022] PUT /identity/v3/projects/c166cdab2dab4e0ab3a5d090ebfaa772/users/7bfd21ee1c504996943465b7158e6b19/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:46.813510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1274/2523] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:46 2022] PUT /identity/v3/projects/3c12fc2d8be14b32ae64f1f8797e1311/users/1b2c6a31169f404699d9ab5309b71c72/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:46.823739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e4e0a-4942-4873-acda-0f4665bf4415 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.824033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e4e0a-4942-4873-acda-0f4665bf4415 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.824303 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e4e0a-4942-4873-acda-0f4665bf4415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.825861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099c01-90dc-4833-934b-b77ac4400641 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:46.826150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099c01-90dc-4833-934b-b77ac4400641 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:46.826427 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099c01-90dc-4833-934b-b77ac4400641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:46.853117 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-095e4e0a-4942-4873-acda-0f4665bf4415 None None] MFA Rules not processed for user `7bfd21ee1c504996943465b7158e6b19`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:46.855321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-095e4e0a-4942-4873-acda-0f4665bf4415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.859305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-40099c01-90dc-4833-934b-b77ac4400641 None None] MFA Rules not processed for user `1b2c6a31169f404699d9ab5309b71c72`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:46.861181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-40099c01-90dc-4833-934b-b77ac4400641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.951924 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1275/2524] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:27:46.952750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40099c01-90dc-4833-934b-b77ac4400641 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.953665 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1250/2525] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:46 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:46.966455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e4e0a-4942-4873-acda-0f4665bf4415 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:46.967723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-095e4e0a-4942-4873-acda-0f4665bf4415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:46.973970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:47.030331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40099c01-90dc-4833-934b-b77ac4400641 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.035456 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c29b03-cbd9-4451-b610-e5e3c0ac3b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.036375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c29b03-cbd9-4451-b610-e5e3c0ac3b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.036659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c29b03-cbd9-4451-b610-e5e3c0ac3b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.036934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c29b03-cbd9-4451-b610-e5e3c0ac3b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.037629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c29b03-cbd9-4451-b610-e5e3c0ac3b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.039583 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c29b03-cbd9-4451-b610-e5e3c0ac3b2a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.050819 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e4e0a-4942-4873-acda-0f4665bf4415 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.054509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2ff80-8239-4b0d-ac39-cabd746aa414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.055161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ff80-8239-4b0d-ac39-cabd746aa414 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.055448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ff80-8239-4b0d-ac39-cabd746aa414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.055724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ff80-8239-4b0d-ac39-cabd746aa414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.067096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2ff80-8239-4b0d-ac39-cabd746aa414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.068556 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2ff80-8239-4b0d-ac39-cabd746aa414 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.070390 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1276/2526] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:46 2022] POST /identity/v3/projects => generated 399 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:47.074865 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c29b03-cbd9-4451-b610-e5e3c0ac3b2a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.085128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1251/2527] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:46 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:27:47.104001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2ff80-8239-4b0d-ac39-cabd746aa414 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.106250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-83e2ff80-8239-4b0d-ac39-cabd746aa414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:47.128870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2ff80-8239-4b0d-ac39-cabd746aa414 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.157143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:47.163365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c29b03-cbd9-4451-b610-e5e3c0ac3b2a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.176371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c320ddf-f47d-4b4d-962c-08c52b59e4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.177107 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c320ddf-f47d-4b4d-962c-08c52b59e4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.177394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c320ddf-f47d-4b4d-962c-08c52b59e4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.177673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c320ddf-f47d-4b4d-962c-08c52b59e4a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.178396 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c320ddf-f47d-4b4d-962c-08c52b59e4a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.180411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c320ddf-f47d-4b4d-962c-08c52b59e4a9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.201219 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.204668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1277/2528] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:47 2022] POST /identity/v3/users => generated 306 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:47.204818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.204818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.204818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] PATH_INFO: `/v3/projects/8f1295403805438d82a585fb1abd9c2a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.205084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c320ddf-f47d-4b4d-962c-08c52b59e4a9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.205169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f1295403805438d82a585fb1abd9c2a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.205873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.254973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f1295403805438d82a585fb1abd9c2a', 'request_id': 'req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:47.260074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c320ddf-f47d-4b4d-962c-08c52b59e4a9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.260691 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f1295403805438d82a585fb1abd9c2a', 'request_id': 'req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:47.268059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f1295403805438d82a585fb1abd9c2a', 'request_id': 'req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:47.281594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] The token cache is being invalidate because project 8f1295403805438d82a585fb1abd9c2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:47.281842 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:47.282491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:47.284055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1252/2529] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:47 2022] DELETE /identity/v3/projects/8f1295403805438d82a585fb1abd9c2a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:47.284821 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.286431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:47.315028 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00814c0-db25-45c2-9916-dca7110b08e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.315694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00814c0-db25-45c2-9916-dca7110b08e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.315982 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00814c0-db25-45c2-9916-dca7110b08e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.316285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00814c0-db25-45c2-9916-dca7110b08e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.318247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f00814c0-db25-45c2-9916-dca7110b08e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:47.327399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:47.358444 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f9a43-5bcc-4175-9f8e-d375d77ab154 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.363583 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797aaee2-796a-4aff-a2d7-30958293a4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.364064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aaee2-796a-4aff-a2d7-30958293a4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.364357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aaee2-796a-4aff-a2d7-30958293a4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.364637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aaee2-796a-4aff-a2d7-30958293a4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.365358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797aaee2-796a-4aff-a2d7-30958293a4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.366997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797aaee2-796a-4aff-a2d7-30958293a4c6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.377444 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1253/2530] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:47.378710 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9cadb-3257-460b-8ff6-d128f14bbea3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.379004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9cadb-3257-460b-8ff6-d128f14bbea3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.379274 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9cadb-3257-460b-8ff6-d128f14bbea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.381629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00814c0-db25-45c2-9916-dca7110b08e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.383112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00814c0-db25-45c2-9916-dca7110b08e9 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.393244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1278/2531] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:47 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:47.393905 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00814c0-db25-45c2-9916-dca7110b08e9 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.398139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-90d9cadb-3257-460b-8ff6-d128f14bbea3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:47.399218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-90d9cadb-3257-460b-8ff6-d128f14bbea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:47.405627 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00814c0-db25-45c2-9916-dca7110b08e9 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.409097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1254/2532] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:47.411966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84873a6-2c06-4e51-85b2-8e2ced0b5ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.412867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84873a6-2c06-4e51-85b2-8e2ced0b5ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.413152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84873a6-2c06-4e51-85b2-8e2ced0b5ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.414271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84873a6-2c06-4e51-85b2-8e2ced0b5ea7 admin admin] PATH_INFO: `/v3/projects/40cc64d2f4e8415499ba3288cf323489/users/f39df5ef0de74475b4ab69d1b6468f23/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.417616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f8897-d576-48ea-a4d5-baa55ac6d4e3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.417996 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f8897-d576-48ea-a4d5-baa55ac6d4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.418272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f8897-d576-48ea-a4d5-baa55ac6d4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.438686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84873a6-2c06-4e51-85b2-8e2ced0b5ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cc64d2f4e8415499ba3288cf323489, user_id=f39df5ef0de74475b4ab69d1b6468f23, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.441511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84873a6-2c06-4e51-85b2-8e2ced0b5ea7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.445015 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8f5f8897-d576-48ea-a4d5-baa55ac6d4e3 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:47.447017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8f5f8897-d576-48ea-a4d5-baa55ac6d4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:47.457911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1255/2533] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:47.458075 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1279/2534] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:47 2022] PUT /identity/v3/projects/40cc64d2f4e8415499ba3288cf323489/users/f39df5ef0de74475b4ab69d1b6468f23/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:47.462987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f8897-d576-48ea-a4d5-baa55ac6d4e3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.465833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84873a6-2c06-4e51-85b2-8e2ced0b5ea7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.473788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f8897-d576-48ea-a4d5-baa55ac6d4e3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.501651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:47.542348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84873a6-2c06-4e51-85b2-8e2ced0b5ea7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.549009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b3cd-2e27-4eb8-ae22-db55a28b2c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.549897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b3cd-2e27-4eb8-ae22-db55a28b2c9e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.550316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b3cd-2e27-4eb8-ae22-db55a28b2c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.550600 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b3cd-2e27-4eb8-ae22-db55a28b2c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.551370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9e09d-f5cf-4c43-abad-b88a2983c4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.551489 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8b3cd-2e27-4eb8-ae22-db55a28b2c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.552046 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9e09d-f5cf-4c43-abad-b88a2983c4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.552329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9e09d-f5cf-4c43-abad-b88a2983c4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.552605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9e09d-f5cf-4c43-abad-b88a2983c4a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.553393 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9e09d-f5cf-4c43-abad-b88a2983c4a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:27:47.553644 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8b3cd-2e27-4eb8-ae22-db55a28b2c9e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.553797 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9e09d-f5cf-4c43-abad-b88a2983c4a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.555744 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9e09d-f5cf-4c43-abad-b88a2983c4a7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.567276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1280/2535] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:27:47.571290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b3cd-2e27-4eb8-ae22-db55a28b2c9e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.572651 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1256/2536] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:27:47.577607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9e09d-f5cf-4c43-abad-b88a2983c4a7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.583236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b3cd-2e27-4eb8-ae22-db55a28b2c9e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.587525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9e09d-f5cf-4c43-abad-b88a2983c4a7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.588036 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669acc8b-5fcb-4269-b5ce-d3eafdc645dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.589321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669acc8b-5fcb-4269-b5ce-d3eafdc645dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.589599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669acc8b-5fcb-4269-b5ce-d3eafdc645dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.589933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669acc8b-5fcb-4269-b5ce-d3eafdc645dc admin admin] PATH_INFO: `/v3/projects/40cc64d2f4e8415499ba3288cf323489/users/f39df5ef0de74475b4ab69d1b6468f23/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.591868 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669acc8b-5fcb-4269-b5ce-d3eafdc645dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cc64d2f4e8415499ba3288cf323489, user_id=f39df5ef0de74475b4ab69d1b6468f23, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.592735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5a1aa-13ac-4979-8ec8-b2afec010d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.593431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5a1aa-13ac-4979-8ec8-b2afec010d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.593743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5a1aa-13ac-4979-8ec8-b2afec010d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.594023 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5a1aa-13ac-4979-8ec8-b2afec010d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.594560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669acc8b-5fcb-4269-b5ce-d3eafdc645dc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.594690 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5a1aa-13ac-4979-8ec8-b2afec010d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.596607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5a1aa-13ac-4979-8ec8-b2afec010d0c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.605649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1281/2537] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:47 2022] PUT /identity/v3/projects/40cc64d2f4e8415499ba3288cf323489/users/f39df5ef0de74475b4ab69d1b6468f23/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:27:47.616542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307c56a-3c6f-412e-82c7-615df57877e3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.616932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307c56a-3c6f-412e-82c7-615df57877e3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.617472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307c56a-3c6f-412e-82c7-615df57877e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.625851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1257/2538] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:47 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:27:47.629692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5a1aa-13ac-4979-8ec8-b2afec010d0c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.644864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2307c56a-3c6f-412e-82c7-615df57877e3 None None] MFA Rules not processed for user `f39df5ef0de74475b4ab69d1b6468f23`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:47.646197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2307c56a-3c6f-412e-82c7-615df57877e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:47.692956 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5a1aa-13ac-4979-8ec8-b2afec010d0c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.697618 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8ca2-604a-4d26-b2d2-65d129e7f6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.698467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8ca2-604a-4d26-b2d2-65d129e7f6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.698933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8ca2-604a-4d26-b2d2-65d129e7f6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.699212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8ca2-604a-4d26-b2d2-65d129e7f6b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.699917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace8ca2-604a-4d26-b2d2-65d129e7f6b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.701853 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace8ca2-604a-4d26-b2d2-65d129e7f6b3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.717198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1282/2539] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:47.726402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1258/2540] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:47 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:27:47.727511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307c56a-3c6f-412e-82c7-615df57877e3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.728101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8ca2-604a-4d26-b2d2-65d129e7f6b3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.729933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6ace8ca2-604a-4d26-b2d2-65d129e7f6b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:47.737679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307c56a-3c6f-412e-82c7-615df57877e3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.742620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f34a5-48a5-4d97-b32f-0685e58e6e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.743572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f34a5-48a5-4d97-b32f-0685e58e6e9c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.743572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f34a5-48a5-4d97-b32f-0685e58e6e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.743882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f34a5-48a5-4d97-b32f-0685e58e6e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.744599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f34a5-48a5-4d97-b32f-0685e58e6e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.747142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f34a5-48a5-4d97-b32f-0685e58e6e9c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.752325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:47.757518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1283/2541] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:47.762348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f34a5-48a5-4d97-b32f-0685e58e6e9c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.773178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f34a5-48a5-4d97-b32f-0685e58e6e9c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.777869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3265b2-2541-4a5a-ad63-f000bf197dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.778518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3265b2-2541-4a5a-ad63-f000bf197dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.778800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3265b2-2541-4a5a-ad63-f000bf197dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.779097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3265b2-2541-4a5a-ad63-f000bf197dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.779835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3265b2-2541-4a5a-ad63-f000bf197dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.781417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3265b2-2541-4a5a-ad63-f000bf197dc2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.790835 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8ca2-604a-4d26-b2d2-65d129e7f6b3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.791840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1284/2542] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:47.795746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3265b2-2541-4a5a-ad63-f000bf197dc2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.796592 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b27f3-4185-47ea-bc1c-5086d66446fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.797294 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b27f3-4185-47ea-bc1c-5086d66446fe service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.797585 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b27f3-4185-47ea-bc1c-5086d66446fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.797951 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b27f3-4185-47ea-bc1c-5086d66446fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.805016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3265b2-2541-4a5a-ad63-f000bf197dc2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.807779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b27f3-4185-47ea-bc1c-5086d66446fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.809254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b27f3-4185-47ea-bc1c-5086d66446fe service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.810572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7e743-45a9-41b4-86ca-1fef5c828e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.811421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e743-45a9-41b4-86ca-1fef5c828e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.811705 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e743-45a9-41b4-86ca-1fef5c828e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.811998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e743-45a9-41b4-86ca-1fef5c828e3d admin admin] PATH_INFO: `/v3/projects/51ea121f7f6a47a2a91b4bac19366643/users/868942ac71b64e83b08a21d2b699859b/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.818886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1259/2543] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:47 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:27:47.827453 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7e743-45a9-41b4-86ca-1fef5c828e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ea121f7f6a47a2a91b4bac19366643, user_id=868942ac71b64e83b08a21d2b699859b, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:47.830498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7e743-45a9-41b4-86ca-1fef5c828e3d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:47.845236 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1285/2544] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:47 2022] PUT /identity/v3/projects/51ea121f7f6a47a2a91b4bac19366643/users/868942ac71b64e83b08a21d2b699859b/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:27:47.856153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700228f0-1259-4897-9a1e-ae37924df07a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.856448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700228f0-1259-4897-9a1e-ae37924df07a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.856733 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700228f0-1259-4897-9a1e-ae37924df07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.884573 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-700228f0-1259-4897-9a1e-ae37924df07a None None] MFA Rules not processed for user `868942ac71b64e83b08a21d2b699859b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:47.885845 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-700228f0-1259-4897-9a1e-ae37924df07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:47.903237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7e743-45a9-41b4-86ca-1fef5c828e3d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.952652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1260/2545] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:47 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:47.957083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700228f0-1259-4897-9a1e-ae37924df07a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:47.962029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7e743-45a9-41b4-86ca-1fef5c828e3d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:47.965914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf72e18-c948-49d0-b0ef-257a8b7e90ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:47.966566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf72e18-c948-49d0-b0ef-257a8b7e90ed service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:47.966848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf72e18-c948-49d0-b0ef-257a8b7e90ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:47.967122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf72e18-c948-49d0-b0ef-257a8b7e90ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:47.968941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0bf72e18-c948-49d0-b0ef-257a8b7e90ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:47.975138 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:48.022166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700228f0-1259-4897-9a1e-ae37924df07a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:48.026872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bb650-75b0-42af-bf47-78a1e018b3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:48.027538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bb650-75b0-42af-bf47-78a1e018b3eb admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:48.027824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bb650-75b0-42af-bf47-78a1e018b3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:48.028101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bb650-75b0-42af-bf47-78a1e018b3eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:48.028806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bb650-75b0-42af-bf47-78a1e018b3eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:48.029788 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf72e18-c948-49d0-b0ef-257a8b7e90ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:48.030763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bb650-75b0-42af-bf47-78a1e018b3eb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:48.031235 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf72e18-c948-49d0-b0ef-257a8b7e90ed service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:48.040703 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1286/2546] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:27:48.056936 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1261/2547] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:47 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:27:48.061749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf72e18-c948-49d0-b0ef-257a8b7e90ed service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:48.072648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf72e18-c948-49d0-b0ef-257a8b7e90ed service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:48.134711 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f4c18-de95-4a64-b038-8c8f06b5c52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:48.135473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f4c18-de95-4a64-b038-8c8f06b5c52a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:48.135760 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f4c18-de95-4a64-b038-8c8f06b5c52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:48.136326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f4c18-de95-4a64-b038-8c8f06b5c52a admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:48.137084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f4c18-de95-4a64-b038-8c8f06b5c52a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:48.139244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f4c18-de95-4a64-b038-8c8f06b5c52a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:48.160914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1287/2548] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:48 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:48.165900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bb650-75b0-42af-bf47-78a1e018b3eb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:48.179226 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bb650-75b0-42af-bf47-78a1e018b3eb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:48.185486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fa106-8bca-4329-a6d8-1186cdcabb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:48.186623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fa106-8bca-4329-a6d8-1186cdcabb12 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:48.186979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fa106-8bca-4329-a6d8-1186cdcabb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:48.187371 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fa106-8bca-4329-a6d8-1186cdcabb12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:48.188300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237fa106-8bca-4329-a6d8-1186cdcabb12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:48.190786 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237fa106-8bca-4329-a6d8-1186cdcabb12 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:48.205923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1262/2549] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:48.210505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f4c18-de95-4a64-b038-8c8f06b5c52a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:48.221735 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f4c18-de95-4a64-b038-8c8f06b5c52a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:48.226487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee7be7-e7f5-4142-a6a9-361361c3c92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:48.227407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee7be7-e7f5-4142-a6a9-361361c3c92b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:48.227699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee7be7-e7f5-4142-a6a9-361361c3c92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:48.227974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee7be7-e7f5-4142-a6a9-361361c3c92b admin admin] PATH_INFO: `/v3/projects/bafdd11d542c40b0b113d47ad8595a70/users/b90a820b150c46c3a1bc2c9de3bcd359/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:48.243668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee7be7-e7f5-4142-a6a9-361361c3c92b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bafdd11d542c40b0b113d47ad8595a70, user_id=b90a820b150c46c3a1bc2c9de3bcd359, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:48.246704 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee7be7-e7f5-4142-a6a9-361361c3c92b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:48.261954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1288/2550] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:48 2022] PUT /identity/v3/projects/bafdd11d542c40b0b113d47ad8595a70/users/b90a820b150c46c3a1bc2c9de3bcd359/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:48.266070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fa106-8bca-4329-a6d8-1186cdcabb12 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:48.278166 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fa106-8bca-4329-a6d8-1186cdcabb12 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:48.306663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:48.345156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb5f04-b6b1-4b45-ab15-d0603b5f0ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:48.345832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb5f04-b6b1-4b45-ab15-d0603b5f0ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:48.346148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb5f04-b6b1-4b45-ab15-d0603b5f0ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:48.346493 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb5f04-b6b1-4b45-ab15-d0603b5f0ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:48.347092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb5f04-b6b1-4b45-ab15-d0603b5f0ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:48.348681 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb5f04-b6b1-4b45-ab15-d0603b5f0ba1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:48.361890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1263/2551] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:48.363257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee7be7-e7f5-4142-a6a9-361361c3c92b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:48.373856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee7be7-e7f5-4142-a6a9-361361c3c92b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:48.378560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4444d79-d559-42aa-86b8-fed7d2c6dab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:48.379496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4444d79-d559-42aa-86b8-fed7d2c6dab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:48.379783 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4444d79-d559-42aa-86b8-fed7d2c6dab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:48.380119 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4444d79-d559-42aa-86b8-fed7d2c6dab5 admin admin] PATH_INFO: `/v3/projects/bafdd11d542c40b0b113d47ad8595a70/users/b90a820b150c46c3a1bc2c9de3bcd359/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:48.382157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4444d79-d559-42aa-86b8-fed7d2c6dab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bafdd11d542c40b0b113d47ad8595a70, user_id=b90a820b150c46c3a1bc2c9de3bcd359, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:48.384803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4444d79-d559-42aa-86b8-fed7d2c6dab5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:48.396635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1289/2552] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:48 2022] PUT /identity/v3/projects/bafdd11d542c40b0b113d47ad8595a70/users/b90a820b150c46c3a1bc2c9de3bcd359/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:48.408719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ee974-423c-4263-8c79-1539d6ce373c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:48.409014 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ee974-423c-4263-8c79-1539d6ce373c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:48.409296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ee974-423c-4263-8c79-1539d6ce373c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:48.422437 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4444d79-d559-42aa-86b8-fed7d2c6dab5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:48.445223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e33ee974-423c-4263-8c79-1539d6ce373c None None] MFA Rules not processed for user `b90a820b150c46c3a1bc2c9de3bcd359`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:48.447197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e33ee974-423c-4263-8c79-1539d6ce373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:48.483611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4444d79-d559-42aa-86b8-fed7d2c6dab5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:48.487339 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d4e7e-34a1-477a-ae58-c84c45f35ed6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:48.487990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d4e7e-34a1-477a-ae58-c84c45f35ed6 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:48.488277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d4e7e-34a1-477a-ae58-c84c45f35ed6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:48.488564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d4e7e-34a1-477a-ae58-c84c45f35ed6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:48.490528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ab0d4e7e-34a1-477a-ae58-c84c45f35ed6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:48.504961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:48.520284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1264/2553] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:48 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:48.532336 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ee974-423c-4263-8c79-1539d6ce373c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:48.533695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e33ee974-423c-4263-8c79-1539d6ce373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:48.545795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ee974-423c-4263-8c79-1539d6ce373c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:48.549413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d97e8-3315-43f5-ae4f-1cac48495535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:48.550090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d97e8-3315-43f5-ae4f-1cac48495535 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:48.550374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d97e8-3315-43f5-ae4f-1cac48495535 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:48.550649 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d97e8-3315-43f5-ae4f-1cac48495535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:48.554830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d4e7e-34a1-477a-ae58-c84c45f35ed6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:48.556517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d4e7e-34a1-477a-ae58-c84c45f35ed6 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:48.562463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d97e8-3315-43f5-ae4f-1cac48495535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:48.563904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d97e8-3315-43f5-ae4f-1cac48495535 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:48.567870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1290/2554] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:27:48.574988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1265/2555] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:48 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:27:49.085739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d4e7e-34a1-477a-ae58-c84c45f35ed6 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:49.147506 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d4e7e-34a1-477a-ae58-c84c45f35ed6 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:49.152746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:49.153608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:49.153989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:49.154271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] PATH_INFO: `/v3/users/868942ac71b64e83b08a21d2b699859b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:49.155458 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] RBAC: Authorizing `identity:delete_user(user_id=868942ac71b64e83b08a21d2b699859b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:49.157382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:49.203885 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868942ac71b64e83b08a21d2b699859b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.204526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868942ac71b64e83b08a21d2b699859b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.205125 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868942ac71b64e83b08a21d2b699859b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.217070 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868942ac71b64e83b08a21d2b699859b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.223839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868942ac71b64e83b08a21d2b699859b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.236681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1291/2556] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:49 2022] DELETE /identity/v3/users/868942ac71b64e83b08a21d2b699859b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:49.241253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5697a04-ee16-4d56-8d2b-a284e3270994 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:49.241545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5697a04-ee16-4d56-8d2b-a284e3270994 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:49.241853 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5697a04-ee16-4d56-8d2b-a284e3270994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:49.259364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c5697a04-ee16-4d56-8d2b-a284e3270994 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:49.261205 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c5697a04-ee16-4d56-8d2b-a284e3270994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:49.320364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1266/2557] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:49.330623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:49.331984 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:49.371184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:49.396585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e46c9-8d11-434b-996b-ece47c6af97a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:49.400448 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82d915-6a75-4ac3-a1b1-bec689f83ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:49.401148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d915-6a75-4ac3-a1b1-bec689f83ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:49.401500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d915-6a75-4ac3-a1b1-bec689f83ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:49.401808 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d915-6a75-4ac3-a1b1-bec689f83ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:49.410965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82d915-6a75-4ac3-a1b1-bec689f83ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:49.412643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82d915-6a75-4ac3-a1b1-bec689f83ee7 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:49.422590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1292/2558] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:49.707903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5697a04-ee16-4d56-8d2b-a284e3270994 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:49.709423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c5697a04-ee16-4d56-8d2b-a284e3270994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:49.726506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5697a04-ee16-4d56-8d2b-a284e3270994 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:49.733099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:49.734078 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:49.734368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:49.734541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] PATH_INFO: `/v3/projects/51ea121f7f6a47a2a91b4bac19366643` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:49.735742 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ea121f7f6a47a2a91b4bac19366643)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:49.737668 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:49.774351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:49.788968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ea121f7f6a47a2a91b4bac19366643', 'request_id': 'req-5cddfbab-225b-412d-abb2-8d1ae97ce69b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.795185 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ea121f7f6a47a2a91b4bac19366643', 'request_id': 'req-5cddfbab-225b-412d-abb2-8d1ae97ce69b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.801423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ea121f7f6a47a2a91b4bac19366643', 'request_id': 'req-5cddfbab-225b-412d-abb2-8d1ae97ce69b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.814925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] The token cache is being invalidate because project 51ea121f7f6a47a2a91b4bac19366643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:49.815223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.815855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.817454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1267/2559] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:49 2022] DELETE /identity/v3/projects/51ea121f7f6a47a2a91b4bac19366643 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:49.823086 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82d915-6a75-4ac3-a1b1-bec689f83ee7 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:49.833535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82d915-6a75-4ac3-a1b1-bec689f83ee7 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:49.892645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ec806-c1d2-4616-b13d-bc02c09e5f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:49.893528 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec806-c1d2-4616-b13d-bc02c09e5f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:49.893840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec806-c1d2-4616-b13d-bc02c09e5f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:49.894116 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec806-c1d2-4616-b13d-bc02c09e5f55 admin admin] PATH_INFO: `/v3/users/b90a820b150c46c3a1bc2c9de3bcd359` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:49.895311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ec806-c1d2-4616-b13d-bc02c09e5f55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b90a820b150c46c3a1bc2c9de3bcd359)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:49.897258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ec806-c1d2-4616-b13d-bc02c09e5f55 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:49.947307 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-722ec806-c1d2-4616-b13d-bc02c09e5f55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b90a820b150c46c3a1bc2c9de3bcd359'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.948023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-722ec806-c1d2-4616-b13d-bc02c09e5f55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b90a820b150c46c3a1bc2c9de3bcd359'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.948823 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-722ec806-c1d2-4616-b13d-bc02c09e5f55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b90a820b150c46c3a1bc2c9de3bcd359'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.961753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-722ec806-c1d2-4616-b13d-bc02c09e5f55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b90a820b150c46c3a1bc2c9de3bcd359'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.969771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-722ec806-c1d2-4616-b13d-bc02c09e5f55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b90a820b150c46c3a1bc2c9de3bcd359'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:49.983239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1293/2560] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:49 2022] DELETE /identity/v3/users/b90a820b150c46c3a1bc2c9de3bcd359 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:50.274670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:50.277063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:50.290383 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddfbab-225b-412d-abb2-8d1ae97ce69b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:50.349298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:50.350134 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:50.350416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:50.350693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] PATH_INFO: `/v3/projects/bafdd11d542c40b0b113d47ad8595a70` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:50.351864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bafdd11d542c40b0b113d47ad8595a70)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:50.353862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:50.399345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bafdd11d542c40b0b113d47ad8595a70', 'request_id': 'req-32dfd351-5d3c-4ab7-88ca-a89c7d942424'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:50.405494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bafdd11d542c40b0b113d47ad8595a70', 'request_id': 'req-32dfd351-5d3c-4ab7-88ca-a89c7d942424'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:50.412410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bafdd11d542c40b0b113d47ad8595a70', 'request_id': 'req-32dfd351-5d3c-4ab7-88ca-a89c7d942424'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:50.424848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] The token cache is being invalidate because project bafdd11d542c40b0b113d47ad8595a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:50.425141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:50.425919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-32dfd351-5d3c-4ab7-88ca-a89c7d942424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:50.427517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1268/2561] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:50 2022] DELETE /identity/v3/projects/bafdd11d542c40b0b113d47ad8595a70 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:50.432736 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36962c9b-e459-4046-aee8-68d49fa76ab9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:50.433230 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36962c9b-e459-4046-aee8-68d49fa76ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:50.433514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36962c9b-e459-4046-aee8-68d49fa76ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:50.452140 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-36962c9b-e459-4046-aee8-68d49fa76ab9 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:50.453344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-36962c9b-e459-4046-aee8-68d49fa76ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:50.506048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:50.524298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1294/2562] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:50.533027 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb04d-a54f-4307-bee3-c7cb58da7234 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:50.533320 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb04d-a54f-4307-bee3-c7cb58da7234 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:50.533599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb04d-a54f-4307-bee3-c7cb58da7234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:50.552199 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-861bb04d-a54f-4307-bee3-c7cb58da7234 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:50.553397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-861bb04d-a54f-4307-bee3-c7cb58da7234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:50.561263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1269/2563] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:50.565311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36962c9b-e459-4046-aee8-68d49fa76ab9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:50.575415 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36962c9b-e459-4046-aee8-68d49fa76ab9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:50.580031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e2792-3123-40cb-8962-b74f38cc4428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:50.580707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e2792-3123-40cb-8962-b74f38cc4428 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:50.580991 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e2792-3123-40cb-8962-b74f38cc4428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:50.581267 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e2792-3123-40cb-8962-b74f38cc4428 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:50.582027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e2792-3123-40cb-8962-b74f38cc4428 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:27:50.582337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e2792-3123-40cb-8962-b74f38cc4428 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:50.583927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e2792-3123-40cb-8962-b74f38cc4428 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:50.596002 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1295/2564] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:27:50.600378 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bb04d-a54f-4307-bee3-c7cb58da7234 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:50.609477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bb04d-a54f-4307-bee3-c7cb58da7234 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:50.614169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea81bce-6e85-4e0b-aa09-f7c5089c5e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:50.614815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea81bce-6e85-4e0b-aa09-f7c5089c5e17 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:50.615099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea81bce-6e85-4e0b-aa09-f7c5089c5e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:50.615376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea81bce-6e85-4e0b-aa09-f7c5089c5e17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:50.616062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea81bce-6e85-4e0b-aa09-f7c5089c5e17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:50.617999 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea81bce-6e85-4e0b-aa09-f7c5089c5e17 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:50.642849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1270/2565] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:50 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:50.646881 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e2792-3123-40cb-8962-b74f38cc4428 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:50.657360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e2792-3123-40cb-8962-b74f38cc4428 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:50.718283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd2605-3eee-41cb-9866-b8f1967c7d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:50.719083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd2605-3eee-41cb-9866-b8f1967c7d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:50.719381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd2605-3eee-41cb-9866-b8f1967c7d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:50.719663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd2605-3eee-41cb-9866-b8f1967c7d6b admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:50.720349 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd2605-3eee-41cb-9866-b8f1967c7d6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:50.722455 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd2605-3eee-41cb-9866-b8f1967c7d6b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:50.746803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1296/2566] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:50 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:50.747988 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea81bce-6e85-4e0b-aa09-f7c5089c5e17 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:50.759169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea81bce-6e85-4e0b-aa09-f7c5089c5e17 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:50.765904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3336a-4279-4a7e-9460-92ec2a4f24fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:50.765904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3336a-4279-4a7e-9460-92ec2a4f24fb admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:50.766125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3336a-4279-4a7e-9460-92ec2a4f24fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:50.766254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3336a-4279-4a7e-9460-92ec2a4f24fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:50.767063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3336a-4279-4a7e-9460-92ec2a4f24fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:50.768743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3336a-4279-4a7e-9460-92ec2a4f24fb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:50.779985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1271/2567] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:50.783733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd2605-3eee-41cb-9866-b8f1967c7d6b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:50.794835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd2605-3eee-41cb-9866-b8f1967c7d6b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:50.800801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca3a14-9e7b-4813-97ef-ce145b96ff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:50.801477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca3a14-9e7b-4813-97ef-ce145b96ff63 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:50.801962 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca3a14-9e7b-4813-97ef-ce145b96ff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:50.802482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca3a14-9e7b-4813-97ef-ce145b96ff63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:50.803926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca3a14-9e7b-4813-97ef-ce145b96ff63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:50.806363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca3a14-9e7b-4813-97ef-ce145b96ff63 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:50.821731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1297/2568] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:50.824660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3336a-4279-4a7e-9460-92ec2a4f24fb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:50.838656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3336a-4279-4a7e-9460-92ec2a4f24fb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:50.848233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764d7f0-4463-4fe6-8bfb-b1fa2c9ddc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:50.848855 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764d7f0-4463-4fe6-8bfb-b1fa2c9ddc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:50.849143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764d7f0-4463-4fe6-8bfb-b1fa2c9ddc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:50.849418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764d7f0-4463-4fe6-8bfb-b1fa2c9ddc10 admin admin] PATH_INFO: `/v3/projects/1fe1de3697c04df4865c4a145a524231/users/b36eda4f44b9428d9ff648979059fb5b/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:50.852947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:50.867550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764d7f0-4463-4fe6-8bfb-b1fa2c9ddc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe1de3697c04df4865c4a145a524231, user_id=b36eda4f44b9428d9ff648979059fb5b, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:50.870794 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764d7f0-4463-4fe6-8bfb-b1fa2c9ddc10 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:50.883577 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1272/2569] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:50 2022] PUT /identity/v3/projects/1fe1de3697c04df4865c4a145a524231/users/b36eda4f44b9428d9ff648979059fb5b/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:50.889620 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76a05-ecc9-4034-9eae-439227c42ab2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:50.889944 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76a05-ecc9-4034-9eae-439227c42ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:50.890226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76a05-ecc9-4034-9eae-439227c42ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:50.908860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f3f76a05-ecc9-4034-9eae-439227c42ab2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:50.910113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f3f76a05-ecc9-4034-9eae-439227c42ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:50.973212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1298/2570] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:50.983434 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764d7f0-4463-4fe6-8bfb-b1fa2c9ddc10 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:50.985098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3764d7f0-4463-4fe6-8bfb-b1fa2c9ddc10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:51.043392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764d7f0-4463-4fe6-8bfb-b1fa2c9ddc10 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:51.047322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d117be-a304-444e-a3c7-4c3768df01f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:51.047992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d117be-a304-444e-a3c7-4c3768df01f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:51.048329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d117be-a304-444e-a3c7-4c3768df01f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:51.048610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d117be-a304-444e-a3c7-4c3768df01f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:51.058632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d117be-a304-444e-a3c7-4c3768df01f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:51.060096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d117be-a304-444e-a3c7-4c3768df01f5 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:51.069702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1273/2571] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:51.699915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b717300-ea1d-4fc8-8ec2-c6f9e9f45cf9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:51.700445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b717300-ea1d-4fc8-8ec2-c6f9e9f45cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:51.700631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b717300-ea1d-4fc8-8ec2-c6f9e9f45cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:51.728594 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2b717300-ea1d-4fc8-8ec2-c6f9e9f45cf9 None None] MFA Rules not processed for user `b36eda4f44b9428d9ff648979059fb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:51.730241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2b717300-ea1d-4fc8-8ec2-c6f9e9f45cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:51.790162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:51.810535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1299/2572] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:51.821587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d117be-a304-444e-a3c7-4c3768df01f5 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:51.823184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-17d117be-a304-444e-a3c7-4c3768df01f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:51.895837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d117be-a304-444e-a3c7-4c3768df01f5 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:51.899841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777e609-483f-4e5c-8927-5fbd6494abd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:51.900474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777e609-483f-4e5c-8927-5fbd6494abd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:51.900770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777e609-483f-4e5c-8927-5fbd6494abd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:51.901150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777e609-483f-4e5c-8927-5fbd6494abd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:51.910488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777e609-483f-4e5c-8927-5fbd6494abd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:51.912208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777e609-483f-4e5c-8927-5fbd6494abd9 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:51.929853 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1274/2573] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:52.421423 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b717300-ea1d-4fc8-8ec2-c6f9e9f45cf9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:52.422985 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2b717300-ea1d-4fc8-8ec2-c6f9e9f45cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:52.440412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b717300-ea1d-4fc8-8ec2-c6f9e9f45cf9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:52.445241 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:52.446164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:52.446447 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:52.446726 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] PATH_INFO: `/v3/users/1b2c6a31169f404699d9ab5309b71c72` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:52.449121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b2c6a31169f404699d9ab5309b71c72)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:52.450887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:52.503588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b2c6a31169f404699d9ab5309b71c72'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:52.505370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b2c6a31169f404699d9ab5309b71c72'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:52.505370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b2c6a31169f404699d9ab5309b71c72'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:52.517833 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b2c6a31169f404699d9ab5309b71c72'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:52.529998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b2c6a31169f404699d9ab5309b71c72'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:52.544824 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1300/2574] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:52 2022] DELETE /identity/v3/users/1b2c6a31169f404699d9ab5309b71c72 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:52.549671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b408afb-d17c-4357-90c1-c81d5f53bdf9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:52.549997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b408afb-d17c-4357-90c1-c81d5f53bdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:52.550664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b408afb-d17c-4357-90c1-c81d5f53bdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:52.568891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3b408afb-d17c-4357-90c1-c81d5f53bdf9 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:52.570180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3b408afb-d17c-4357-90c1-c81d5f53bdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:52.639348 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1275/2575] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:52.648673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:52.650337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:52.712324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2eb5a-891c-4d4b-869f-49da245681a9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:52.716038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d64630-3f07-4fd9-9ec5-185f5a021e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:52.716694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d64630-3f07-4fd9-9ec5-185f5a021e5b service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:52.716980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d64630-3f07-4fd9-9ec5-185f5a021e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:52.717289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d64630-3f07-4fd9-9ec5-185f5a021e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:52.726935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d64630-3f07-4fd9-9ec5-185f5a021e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:52.728405 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d64630-3f07-4fd9-9ec5-185f5a021e5b service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:52.739134 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1301/2576] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:53.023805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b408afb-d17c-4357-90c1-c81d5f53bdf9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:53.025253 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3b408afb-d17c-4357-90c1-c81d5f53bdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:53.039321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b408afb-d17c-4357-90c1-c81d5f53bdf9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:53.044643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:53.045545 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:53.045877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:53.046159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] PATH_INFO: `/v3/projects/3c12fc2d8be14b32ae64f1f8797e1311` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:53.047351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c12fc2d8be14b32ae64f1f8797e1311)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:53.049278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:53.080201 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:53.098886 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c12fc2d8be14b32ae64f1f8797e1311', 'request_id': 'req-77f903a1-5fb2-4348-afe2-2425faa042b9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.105083 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c12fc2d8be14b32ae64f1f8797e1311', 'request_id': 'req-77f903a1-5fb2-4348-afe2-2425faa042b9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.111197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c12fc2d8be14b32ae64f1f8797e1311', 'request_id': 'req-77f903a1-5fb2-4348-afe2-2425faa042b9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.123516 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] The token cache is being invalidate because project 3c12fc2d8be14b32ae64f1f8797e1311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:53.123810 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.124434 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-77f903a1-5fb2-4348-afe2-2425faa042b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.126018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1276/2577] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:53 2022] DELETE /identity/v3/projects/3c12fc2d8be14b32ae64f1f8797e1311 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:53.130027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d64630-3f07-4fd9-9ec5-185f5a021e5b service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:53.139683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d64630-3f07-4fd9-9ec5-185f5a021e5b service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:53.196016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:53.196819 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:53.197105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:53.197385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] PATH_INFO: `/v3/users/f39df5ef0de74475b4ab69d1b6468f23` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:53.198669 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f39df5ef0de74475b4ab69d1b6468f23)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:53.200592 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:53.205911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:53.246400 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f39df5ef0de74475b4ab69d1b6468f23'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.247038 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f39df5ef0de74475b4ab69d1b6468f23'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.247642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f39df5ef0de74475b4ab69d1b6468f23'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.259851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f39df5ef0de74475b4ab69d1b6468f23'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.268309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f39df5ef0de74475b4ab69d1b6468f23'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.287374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1302/2578] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:27:53 2022] DELETE /identity/v3/users/f39df5ef0de74475b4ab69d1b6468f23 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:53.327771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9757c-53ff-443e-a90b-7eb92335843e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:53.327771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9757c-53ff-443e-a90b-7eb92335843e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:53.327771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9757c-53ff-443e-a90b-7eb92335843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:53.344577 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-faa9757c-53ff-443e-a90b-7eb92335843e None None] MFA Rules not processed for user `b36eda4f44b9428d9ff648979059fb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:53.345805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-faa9757c-53ff-443e-a90b-7eb92335843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:53.407530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1277/2579] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:53.695496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:53.697060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:53.769057 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c53e1b-c097-47df-932e-4ecb010599e2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:53.774016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:53.774888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:53.775174 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:53.775454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] PATH_INFO: `/v3/projects/40cc64d2f4e8415499ba3288cf323489` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:53.776728 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40cc64d2f4e8415499ba3288cf323489)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:53.778692 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:53.827384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40cc64d2f4e8415499ba3288cf323489', 'request_id': 'req-230a155b-f6b8-4eaa-b0de-fcdb85effef0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.833664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40cc64d2f4e8415499ba3288cf323489', 'request_id': 'req-230a155b-f6b8-4eaa-b0de-fcdb85effef0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.841486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40cc64d2f4e8415499ba3288cf323489', 'request_id': 'req-230a155b-f6b8-4eaa-b0de-fcdb85effef0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.859498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] The token cache is being invalidate because project 40cc64d2f4e8415499ba3288cf323489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:27:53.859814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.860517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-230a155b-f6b8-4eaa-b0de-fcdb85effef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:27:53.862152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1303/2580] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:27:53 2022] DELETE /identity/v3/projects/40cc64d2f4e8415499ba3288cf323489 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:53.867100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d680-d479-4428-a364-1628ca7cf255 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:53.867417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d680-d479-4428-a364-1628ca7cf255 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:53.867704 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d680-d479-4428-a364-1628ca7cf255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:53.887389 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a104d680-d479-4428-a364-1628ca7cf255 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:53.888604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a104d680-d479-4428-a364-1628ca7cf255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:53.965479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1278/2581] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:53.967548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6b5d1-cd8e-407e-9b77-e4843a9ee11f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:53.967841 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6b5d1-cd8e-407e-9b77-e4843a9ee11f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:53.968121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6b5d1-cd8e-407e-9b77-e4843a9ee11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:53.987702 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-17a6b5d1-cd8e-407e-9b77-e4843a9ee11f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:53.988890 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-17a6b5d1-cd8e-407e-9b77-e4843a9ee11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:53.997406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1304/2582] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:54.001668 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104d680-d479-4428-a364-1628ca7cf255 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.015737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104d680-d479-4428-a364-1628ca7cf255 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.021927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cbd04-e0ea-4e85-b16c-3391c4bc53f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.022804 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cbd04-e0ea-4e85-b16c-3391c4bc53f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.023096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cbd04-e0ea-4e85-b16c-3391c4bc53f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.023284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cbd04-e0ea-4e85-b16c-3391c4bc53f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.024084 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cbd04-e0ea-4e85-b16c-3391c4bc53f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:27:54.024602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cbd04-e0ea-4e85-b16c-3391c4bc53f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.027170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cbd04-e0ea-4e85-b16c-3391c4bc53f9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.041998 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1279/2583] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:27:54.046790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6b5d1-cd8e-407e-9b77-e4843a9ee11f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.057663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6b5d1-cd8e-407e-9b77-e4843a9ee11f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.062875 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f5662-7f30-4037-a854-42c10ccad5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.063564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f5662-7f30-4037-a854-42c10ccad5dd admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.063852 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f5662-7f30-4037-a854-42c10ccad5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.064129 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f5662-7f30-4037-a854-42c10ccad5dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.064820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f5662-7f30-4037-a854-42c10ccad5dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.066776 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f5662-7f30-4037-a854-42c10ccad5dd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.071287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:54.101289 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1305/2584] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:54 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:54.102479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cbd04-e0ea-4e85-b16c-3391c4bc53f9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.115191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cbd04-e0ea-4e85-b16c-3391c4bc53f9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.163239 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:54.178172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b176290-af70-4321-8bf9-da4e396da930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.178915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b176290-af70-4321-8bf9-da4e396da930 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.179170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b176290-af70-4321-8bf9-da4e396da930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.179491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b176290-af70-4321-8bf9-da4e396da930 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.180213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b176290-af70-4321-8bf9-da4e396da930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.183050 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b176290-af70-4321-8bf9-da4e396da930 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.208132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1280/2585] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:54 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:27:54.209388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f5662-7f30-4037-a854-42c10ccad5dd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.219801 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f5662-7f30-4037-a854-42c10ccad5dd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.225599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c708e-28b7-4686-bdd0-a2b6355f89ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.226283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c708e-28b7-4686-bdd0-a2b6355f89ca admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.226564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c708e-28b7-4686-bdd0-a2b6355f89ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.226838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c708e-28b7-4686-bdd0-a2b6355f89ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.227568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c708e-28b7-4686-bdd0-a2b6355f89ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.229960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c708e-28b7-4686-bdd0-a2b6355f89ca admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.245166 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1306/2586] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:54.247063 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b176290-af70-4321-8bf9-da4e396da930 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.257088 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b176290-af70-4321-8bf9-da4e396da930 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.262144 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68cd3d-893c-4a2f-b79f-703c19e288db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.262929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68cd3d-893c-4a2f-b79f-703c19e288db admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.263221 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68cd3d-893c-4a2f-b79f-703c19e288db admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.263503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68cd3d-893c-4a2f-b79f-703c19e288db admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.264234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68cd3d-893c-4a2f-b79f-703c19e288db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.266447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68cd3d-893c-4a2f-b79f-703c19e288db admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.279210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1281/2587] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:54.283320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c708e-28b7-4686-bdd0-a2b6355f89ca admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.295090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c708e-28b7-4686-bdd0-a2b6355f89ca admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.300172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b115d-1248-4d00-9d7f-9f6f63ef482a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.301081 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b115d-1248-4d00-9d7f-9f6f63ef482a admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.301355 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b115d-1248-4d00-9d7f-9f6f63ef482a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.301654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b115d-1248-4d00-9d7f-9f6f63ef482a admin admin] PATH_INFO: `/v3/projects/1056c5aab2364e3982bd371efa71a907/users/d7f9db7ce3284bcd835c3fd39f9fd343/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.318478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b115d-1248-4d00-9d7f-9f6f63ef482a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1056c5aab2364e3982bd371efa71a907, user_id=d7f9db7ce3284bcd835c3fd39f9fd343, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.321254 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b115d-1248-4d00-9d7f-9f6f63ef482a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.334775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1307/2588] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:54 2022] PUT /identity/v3/projects/1056c5aab2364e3982bd371efa71a907/users/d7f9db7ce3284bcd835c3fd39f9fd343/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:54.347039 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fb385-ad15-411b-8561-9f352d1321ad None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.347475 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fb385-ad15-411b-8561-9f352d1321ad None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.347763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fb385-ad15-411b-8561-9f352d1321ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.390005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a82fb385-ad15-411b-8561-9f352d1321ad None None] MFA Rules not processed for user `d7f9db7ce3284bcd835c3fd39f9fd343`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:54.391532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a82fb385-ad15-411b-8561-9f352d1321ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:54.474455 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1282/2589] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:54.475858 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b115d-1248-4d00-9d7f-9f6f63ef482a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.550756 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b115d-1248-4d00-9d7f-9f6f63ef482a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.555519 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62e8ad-c2da-428a-8c0d-589606551d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.556288 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62e8ad-c2da-428a-8c0d-589606551d62 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.556544 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62e8ad-c2da-428a-8c0d-589606551d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.556827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62e8ad-c2da-428a-8c0d-589606551d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.557560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62e8ad-c2da-428a-8c0d-589606551d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.559662 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62e8ad-c2da-428a-8c0d-589606551d62 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.593224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1308/2590] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:27:54 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:27:54.594386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fb385-ad15-411b-8561-9f352d1321ad None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.608254 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fb385-ad15-411b-8561-9f352d1321ad None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.671913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e238f07-10c1-4174-ae22-1d6243eb0793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.672594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e238f07-10c1-4174-ae22-1d6243eb0793 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.672879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e238f07-10c1-4174-ae22-1d6243eb0793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.673165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e238f07-10c1-4174-ae22-1d6243eb0793 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.673931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e238f07-10c1-4174-ae22-1d6243eb0793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.675854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e238f07-10c1-4174-ae22-1d6243eb0793 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.695505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1283/2591] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:27:54 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:27:54.699897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62e8ad-c2da-428a-8c0d-589606551d62 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.710112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62e8ad-c2da-428a-8c0d-589606551d62 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.715413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15779c23-7452-4524-be0f-1a5245f17bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.716067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779c23-7452-4524-be0f-1a5245f17bed admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.716370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779c23-7452-4524-be0f-1a5245f17bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.717187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779c23-7452-4524-be0f-1a5245f17bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.717960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15779c23-7452-4524-be0f-1a5245f17bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.719781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15779c23-7452-4524-be0f-1a5245f17bed admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.731221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1309/2592] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:54.735159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e238f07-10c1-4174-ae22-1d6243eb0793 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.744994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e238f07-10c1-4174-ae22-1d6243eb0793 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.749734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd745ca-f1ae-4bea-bb42-f0079d8c35a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.750616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd745ca-f1ae-4bea-bb42-f0079d8c35a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.750906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd745ca-f1ae-4bea-bb42-f0079d8c35a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.751182 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd745ca-f1ae-4bea-bb42-f0079d8c35a6 admin admin] PATH_INFO: `/v3/projects/34d1f2056e434b65b6a37a6a59802871/users/d55fa36f7bab43acafee1c565719ad5d/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.766911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd745ca-f1ae-4bea-bb42-f0079d8c35a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d1f2056e434b65b6a37a6a59802871, user_id=d55fa36f7bab43acafee1c565719ad5d, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.769568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd745ca-f1ae-4bea-bb42-f0079d8c35a6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.783594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1284/2593] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:54 2022] PUT /identity/v3/projects/34d1f2056e434b65b6a37a6a59802871/users/d55fa36f7bab43acafee1c565719ad5d/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:27:54.787866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15779c23-7452-4524-be0f-1a5245f17bed admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.798641 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15779c23-7452-4524-be0f-1a5245f17bed admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.853812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035309e-8ab4-4e4a-98e3-f1225b855847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.854535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035309e-8ab4-4e4a-98e3-f1225b855847 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.854820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035309e-8ab4-4e4a-98e3-f1225b855847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.855098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035309e-8ab4-4e4a-98e3-f1225b855847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.855815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035309e-8ab4-4e4a-98e3-f1225b855847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.857406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035309e-8ab4-4e4a-98e3-f1225b855847 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.867010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1310/2594] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:27:54.871032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd745ca-f1ae-4bea-bb42-f0079d8c35a6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:54.881157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd745ca-f1ae-4bea-bb42-f0079d8c35a6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:54.886342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eea24e-8953-44fb-a5bd-b52ba84b8f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:54.887220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eea24e-8953-44fb-a5bd-b52ba84b8f2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.887501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eea24e-8953-44fb-a5bd-b52ba84b8f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.887778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eea24e-8953-44fb-a5bd-b52ba84b8f2c admin admin] PATH_INFO: `/v3/projects/34d1f2056e434b65b6a37a6a59802871/users/d55fa36f7bab43acafee1c565719ad5d/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.889738 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eea24e-8953-44fb-a5bd-b52ba84b8f2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d1f2056e434b65b6a37a6a59802871, user_id=d55fa36f7bab43acafee1c565719ad5d, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:54.892389 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eea24e-8953-44fb-a5bd-b52ba84b8f2c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:54.903373 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1285/2595] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:27:54 2022] PUT /identity/v3/projects/34d1f2056e434b65b6a37a6a59802871/users/d55fa36f7bab43acafee1c565719ad5d/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:27:54.913580 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f8e81-b2ce-47da-996e-054cbfbcc38a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:54.913993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f8e81-b2ce-47da-996e-054cbfbcc38a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:54.914339 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f8e81-b2ce-47da-996e-054cbfbcc38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:54.940923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-247f8e81-b2ce-47da-996e-054cbfbcc38a None None] MFA Rules not processed for user `d55fa36f7bab43acafee1c565719ad5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:54.942185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-247f8e81-b2ce-47da-996e-054cbfbcc38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:54.954359 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:55.080127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1311/2596] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:55.093326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eea24e-8953-44fb-a5bd-b52ba84b8f2c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:55.102619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b9eea24e-8953-44fb-a5bd-b52ba84b8f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:55.118799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:27:55.199062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eea24e-8953-44fb-a5bd-b52ba84b8f2c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:55.206156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f79301-2a87-49cc-9e4a-b3cc036e7829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:55.206901 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f79301-2a87-49cc-9e4a-b3cc036e7829 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:55.207236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f79301-2a87-49cc-9e4a-b3cc036e7829 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:55.207499 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f79301-2a87-49cc-9e4a-b3cc036e7829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:55.283761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f79301-2a87-49cc-9e4a-b3cc036e7829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:55.285340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f79301-2a87-49cc-9e4a-b3cc036e7829 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:55.295372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1286/2597] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:27:55.499856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad0e51-d4ce-4e49-bfd3-90e9253e138b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:55.500149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad0e51-d4ce-4e49-bfd3-90e9253e138b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:55.500435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad0e51-d4ce-4e49-bfd3-90e9253e138b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:55.519060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a6ad0e51-d4ce-4e49-bfd3-90e9253e138b None None] MFA Rules not processed for user `c935f035c757482ca912a8c4b77714a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:27:55.520202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a6ad0e51-d4ce-4e49-bfd3-90e9253e138b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:55.593279 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1312/2598] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:27:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:27:59.770906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f79301-2a87-49cc-9e4a-b3cc036e7829 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:27:59.772370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-73f79301-2a87-49cc-9e4a-b3cc036e7829 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:27:59.784614 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f79301-2a87-49cc-9e4a-b3cc036e7829 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:27:59.788785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33855075-d170-46c1-bc21-04d3dae16e7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:27:59.789429 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33855075-d170-46c1-bc21-04d3dae16e7d service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:27:59.789706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33855075-d170-46c1-bc21-04d3dae16e7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:27:59.790006 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33855075-d170-46c1-bc21-04d3dae16e7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:27:59.800887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33855075-d170-46c1-bc21-04d3dae16e7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:27:59.802345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33855075-d170-46c1-bc21-04d3dae16e7d service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:27:59.813021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1287/2599] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:27:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:04.807447 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad0e51-d4ce-4e49-bfd3-90e9253e138b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:04.808980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a6ad0e51-d4ce-4e49-bfd3-90e9253e138b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:04.876573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad0e51-d4ce-4e49-bfd3-90e9253e138b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:04.881550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055f45e-43a9-40c9-9eea-e6fa9e095717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:04.882462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f45e-43a9-40c9-9eea-e6fa9e095717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:04.882746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f45e-43a9-40c9-9eea-e6fa9e095717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:04.883020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f45e-43a9-40c9-9eea-e6fa9e095717 admin admin] PATH_INFO: `/v3/users/1a0e00f42d39426184c802d087d6c531` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:04.884336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055f45e-43a9-40c9-9eea-e6fa9e095717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a0e00f42d39426184c802d087d6c531)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:04.886297 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055f45e-43a9-40c9-9eea-e6fa9e095717 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:04.908853 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33855075-d170-46c1-bc21-04d3dae16e7d service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:04.922629 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33855075-d170-46c1-bc21-04d3dae16e7d service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:04.927914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e7e98-f86a-4ab0-a403-45db7d4cb912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:04.928897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e7e98-f86a-4ab0-a403-45db7d4cb912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:04.929370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e7e98-f86a-4ab0-a403-45db7d4cb912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:04.929844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e7e98-f86a-4ab0-a403-45db7d4cb912 admin admin] PATH_INFO: `/v3/users/d7f9db7ce3284bcd835c3fd39f9fd343` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:04.931178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e7e98-f86a-4ab0-a403-45db7d4cb912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7f9db7ce3284bcd835c3fd39f9fd343)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:04.933919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e7e98-f86a-4ab0-a403-45db7d4cb912 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:04.941512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a055f45e-43a9-40c9-9eea-e6fa9e095717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a0e00f42d39426184c802d087d6c531'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:04.942308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a055f45e-43a9-40c9-9eea-e6fa9e095717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a0e00f42d39426184c802d087d6c531'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:04.942899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a055f45e-43a9-40c9-9eea-e6fa9e095717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a0e00f42d39426184c802d087d6c531'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:04.948382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:04.957370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a055f45e-43a9-40c9-9eea-e6fa9e095717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a0e00f42d39426184c802d087d6c531'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:04.965684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a055f45e-43a9-40c9-9eea-e6fa9e095717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a0e00f42d39426184c802d087d6c531'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:04.982354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1313/2600] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:04 2022] DELETE /identity/v3/users/1a0e00f42d39426184c802d087d6c531 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:04.986877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8043e01-f674-4744-8871-21af5f90fc0d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:04.987372 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8043e01-f674-4744-8871-21af5f90fc0d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:04.987648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8043e01-f674-4744-8871-21af5f90fc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:04.988124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ae1e7e98-f86a-4ab0-a403-45db7d4cb912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7f9db7ce3284bcd835c3fd39f9fd343'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:04.996297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ae1e7e98-f86a-4ab0-a403-45db7d4cb912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7f9db7ce3284bcd835c3fd39f9fd343'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:04.996911 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ae1e7e98-f86a-4ab0-a403-45db7d4cb912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7f9db7ce3284bcd835c3fd39f9fd343'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.006108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b8043e01-f674-4744-8871-21af5f90fc0d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:05.007410 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b8043e01-f674-4744-8871-21af5f90fc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:05.010645 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ae1e7e98-f86a-4ab0-a403-45db7d4cb912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7f9db7ce3284bcd835c3fd39f9fd343'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.017607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ae1e7e98-f86a-4ab0-a403-45db7d4cb912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7f9db7ce3284bcd835c3fd39f9fd343'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.023653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1288/2601] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:04 2022] DELETE /identity/v3/users/d7f9db7ce3284bcd835c3fd39f9fd343 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:05.028715 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f1725-7854-453a-960b-b9cc82955d43 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:05.029004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f1725-7854-453a-960b-b9cc82955d43 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:05.029596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f1725-7854-453a-960b-b9cc82955d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:05.052770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-985f1725-7854-453a-960b-b9cc82955d43 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:05.054231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-985f1725-7854-453a-960b-b9cc82955d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:05.069547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:05.074460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1314/2602] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:05.085628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8043e01-f674-4744-8871-21af5f90fc0d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:05.088357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b8043e01-f674-4744-8871-21af5f90fc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:05.135778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1289/2603] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:05.145829 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f1725-7854-453a-960b-b9cc82955d43 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:05.151987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8043e01-f674-4744-8871-21af5f90fc0d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:05.204053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f1725-7854-453a-960b-b9cc82955d43 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:05.205168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dac6a0-63b3-4334-865f-62953fb0a01f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:05.205861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dac6a0-63b3-4334-865f-62953fb0a01f service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:05.206149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dac6a0-63b3-4334-865f-62953fb0a01f service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:05.206430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dac6a0-63b3-4334-865f-62953fb0a01f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:05.207931 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8423ccb-0529-4203-b5d2-ed0539641e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:05.208574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8423ccb-0529-4203-b5d2-ed0539641e3b service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:05.208860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8423ccb-0529-4203-b5d2-ed0539641e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:05.209132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8423ccb-0529-4203-b5d2-ed0539641e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:05.217194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dac6a0-63b3-4334-865f-62953fb0a01f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:05.218729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dac6a0-63b3-4334-865f-62953fb0a01f service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:05.220081 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8423ccb-0529-4203-b5d2-ed0539641e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:05.221503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8423ccb-0529-4203-b5d2-ed0539641e3b service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:05.230413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1315/2604] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:28:05.239930 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1290/2605] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:28:05.717331 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dac6a0-63b3-4334-865f-62953fb0a01f service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:05.718781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-75dac6a0-63b3-4334-865f-62953fb0a01f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:05.723400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8423ccb-0529-4203-b5d2-ed0539641e3b service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:05.725390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a8423ccb-0529-4203-b5d2-ed0539641e3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:05.732992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dac6a0-63b3-4334-865f-62953fb0a01f service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:05.737914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:05.738809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:05.739095 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:05.739375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] PATH_INFO: `/v3/projects/b040cd35a96f4f91be5c2a9d8bf9e4d7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:05.747512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b040cd35a96f4f91be5c2a9d8bf9e4d7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:05.749587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:05.763646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8423ccb-0529-4203-b5d2-ed0539641e3b service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:05.775241 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:05.778450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:05.778803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:05.781721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] PATH_INFO: `/v3/projects/1056c5aab2364e3982bd371efa71a907` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:05.783569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1056c5aab2364e3982bd371efa71a907)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:05.785747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:05.793541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:05.817488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b040cd35a96f4f91be5c2a9d8bf9e4d7', 'request_id': 'req-fdbb730e-7575-4150-9902-cbf3eecd4fd6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.825336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b040cd35a96f4f91be5c2a9d8bf9e4d7', 'request_id': 'req-fdbb730e-7575-4150-9902-cbf3eecd4fd6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.835922 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b040cd35a96f4f91be5c2a9d8bf9e4d7', 'request_id': 'req-fdbb730e-7575-4150-9902-cbf3eecd4fd6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.847719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1056c5aab2364e3982bd371efa71a907', 'request_id': 'req-872577f4-775b-448e-b40c-549552ee65f4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.850917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] The token cache is being invalidate because project b040cd35a96f4f91be5c2a9d8bf9e4d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:05.851211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.851873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.853863 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1316/2606] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:05 2022] DELETE /identity/v3/projects/b040cd35a96f4f91be5c2a9d8bf9e4d7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:05.853992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1056c5aab2364e3982bd371efa71a907', 'request_id': 'req-872577f4-775b-448e-b40c-549552ee65f4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.857395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:05.859079 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:05.861161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1056c5aab2364e3982bd371efa71a907', 'request_id': 'req-872577f4-775b-448e-b40c-549552ee65f4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.877980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] The token cache is being invalidate because project 1056c5aab2364e3982bd371efa71a907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:05.878414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.879148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:05.881079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1291/2607] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:05 2022] DELETE /identity/v3/projects/1056c5aab2364e3982bd371efa71a907 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:05.884510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:05.887148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:05.940790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:05.944709 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb730e-7575-4150-9902-cbf3eecd4fd6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:05.959477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872577f4-775b-448e-b40c-549552ee65f4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:05.964859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:05.965788 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:05.966072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:05.966345 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] PATH_INFO: `/v3/users/d55fa36f7bab43acafee1c565719ad5d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:05.967528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d55fa36f7bab43acafee1c565719ad5d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:05.969446 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:06.004083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:06.004967 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.005251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.005525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] PATH_INFO: `/v3/users/7bfd21ee1c504996943465b7158e6b19` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.006762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bfd21ee1c504996943465b7158e6b19)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:06.008692 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:06.017428 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd55fa36f7bab43acafee1c565719ad5d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.025211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd55fa36f7bab43acafee1c565719ad5d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.026827 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd55fa36f7bab43acafee1c565719ad5d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.042813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd55fa36f7bab43acafee1c565719ad5d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.050259 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd55fa36f7bab43acafee1c565719ad5d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.056488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1292/2608] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:05 2022] DELETE /identity/v3/users/d55fa36f7bab43acafee1c565719ad5d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:06.058244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bfd21ee1c504996943465b7158e6b19'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.058935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bfd21ee1c504996943465b7158e6b19'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.059594 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bfd21ee1c504996943465b7158e6b19'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.071055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:06.072809 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bfd21ee1c504996943465b7158e6b19'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.081925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bfd21ee1c504996943465b7158e6b19'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.098487 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1317/2609] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:05 2022] DELETE /identity/v3/users/7bfd21ee1c504996943465b7158e6b19 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:06.391265 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:06.392791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:06.411550 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:06.415541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:06.477908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41285a5-1228-4fe2-8f96-a63b783f2b67 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:06.502915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab15ad-6641-4790-9b5b-13021eba42c6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:06.507510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:06.508376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.508657 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.508938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] PATH_INFO: `/v3/projects/c166cdab2dab4e0ab3a5d090ebfaa772` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.510300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c166cdab2dab4e0ab3a5d090ebfaa772)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:06.512223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:06.537118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:06.538146 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.538425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.538744 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] PATH_INFO: `/v3/projects/34d1f2056e434b65b6a37a6a59802871` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.539974 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d1f2056e434b65b6a37a6a59802871)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:06.543076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:06.561392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:06.562485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c166cdab2dab4e0ab3a5d090ebfaa772', 'request_id': 'req-2644bda5-f1d4-4796-8feb-5bf10229ed3d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.570533 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c166cdab2dab4e0ab3a5d090ebfaa772', 'request_id': 'req-2644bda5-f1d4-4796-8feb-5bf10229ed3d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.579402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c166cdab2dab4e0ab3a5d090ebfaa772', 'request_id': 'req-2644bda5-f1d4-4796-8feb-5bf10229ed3d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.591246 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] The token cache is being invalidate because project c166cdab2dab4e0ab3a5d090ebfaa772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:06.591542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.592202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2644bda5-f1d4-4796-8feb-5bf10229ed3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.593803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1318/2610] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:06 2022] DELETE /identity/v3/projects/c166cdab2dab4e0ab3a5d090ebfaa772 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:06.598730 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d1f2056e434b65b6a37a6a59802871', 'request_id': 'req-33957f32-5bfc-45c1-99e6-e8bab754eaf1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.598990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593920cf-58e4-4682-9b51-cde5d79d4883 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.599208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593920cf-58e4-4682-9b51-cde5d79d4883 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.599455 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593920cf-58e4-4682-9b51-cde5d79d4883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.606301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d1f2056e434b65b6a37a6a59802871', 'request_id': 'req-33957f32-5bfc-45c1-99e6-e8bab754eaf1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.613157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d1f2056e434b65b6a37a6a59802871', 'request_id': 'req-33957f32-5bfc-45c1-99e6-e8bab754eaf1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.621990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-593920cf-58e4-4682-9b51-cde5d79d4883 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:06.623251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-593920cf-58e4-4682-9b51-cde5d79d4883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:06.628536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] The token cache is being invalidate because project 34d1f2056e434b65b6a37a6a59802871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:06.628828 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.629469 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-33957f32-5bfc-45c1-99e6-e8bab754eaf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:06.631106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1293/2611] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:06 2022] DELETE /identity/v3/projects/34d1f2056e434b65b6a37a6a59802871 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:06.638072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45578d68-0997-487a-b03c-8eb0e6641a9b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.638943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45578d68-0997-487a-b03c-8eb0e6641a9b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.639576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45578d68-0997-487a-b03c-8eb0e6641a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.665180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-45578d68-0997-487a-b03c-8eb0e6641a9b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:06.667171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-45578d68-0997-487a-b03c-8eb0e6641a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:06.692621 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1319/2612] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:06.702671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae81e9-d068-46d0-9ae3-589fb1b6e5a5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.703099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae81e9-d068-46d0-9ae3-589fb1b6e5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.703483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae81e9-d068-46d0-9ae3-589fb1b6e5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.726583 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e4ae81e9-d068-46d0-9ae3-589fb1b6e5a5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:06.727891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e4ae81e9-d068-46d0-9ae3-589fb1b6e5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:06.735563 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1294/2613] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:06.739341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1320/2614] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:06.744601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45578d68-0997-487a-b03c-8eb0e6641a9b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:06.745283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d65f49-17d0-431a-9558-c6b0103770fc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.745611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d65f49-17d0-431a-9558-c6b0103770fc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.745989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d65f49-17d0-431a-9558-c6b0103770fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.754285 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:06.756510 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45578d68-0997-487a-b03c-8eb0e6641a9b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:06.764918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf584a-1b8d-4439-a57b-5360662e1019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:06.765865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf584a-1b8d-4439-a57b-5360662e1019 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.766471 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf584a-1b8d-4439-a57b-5360662e1019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.766913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf584a-1b8d-4439-a57b-5360662e1019 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.767927 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b5d65f49-17d0-431a-9558-c6b0103770fc None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:06.768448 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf584a-1b8d-4439-a57b-5360662e1019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:28:06.768773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf584a-1b8d-4439-a57b-5360662e1019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:06.769401 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b5d65f49-17d0-431a-9558-c6b0103770fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:06.770432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf584a-1b8d-4439-a57b-5360662e1019 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:06.780569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1321/2615] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:06.785316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d65f49-17d0-431a-9558-c6b0103770fc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:06.785562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1295/2616] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:28:06.790145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf584a-1b8d-4439-a57b-5360662e1019 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:06.796787 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d65f49-17d0-431a-9558-c6b0103770fc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:06.801540 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92c8ff-ffe7-4529-a529-7f96b9bc4897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:06.802247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c8ff-ffe7-4529-a529-7f96b9bc4897 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.802374 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf584a-1b8d-4439-a57b-5360662e1019 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:06.802532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c8ff-ffe7-4529-a529-7f96b9bc4897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.802839 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c8ff-ffe7-4529-a529-7f96b9bc4897 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.803595 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92c8ff-ffe7-4529-a529-7f96b9bc4897 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:28:06.803909 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92c8ff-ffe7-4529-a529-7f96b9bc4897 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:06.805630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92c8ff-ffe7-4529-a529-7f96b9bc4897 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:06.808022 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b2944-1811-4679-b2c0-32bc1d422674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:06.809359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b2944-1811-4679-b2c0-32bc1d422674 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.809740 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b2944-1811-4679-b2c0-32bc1d422674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.810103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b2944-1811-4679-b2c0-32bc1d422674 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.810962 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b2944-1811-4679-b2c0-32bc1d422674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:06.812933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b2944-1811-4679-b2c0-32bc1d422674 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:06.827663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1322/2617] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:28:06.827663 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92c8ff-ffe7-4529-a529-7f96b9bc4897 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:06.833639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92c8ff-ffe7-4529-a529-7f96b9bc4897 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:06.839154 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f680a48-ecf3-4a5a-9cb2-a1cdfa160aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:06.839870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f680a48-ecf3-4a5a-9cb2-a1cdfa160aac admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.840163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f680a48-ecf3-4a5a-9cb2-a1cdfa160aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.840435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f680a48-ecf3-4a5a-9cb2-a1cdfa160aac admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.841120 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f680a48-ecf3-4a5a-9cb2-a1cdfa160aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:06.843115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f680a48-ecf3-4a5a-9cb2-a1cdfa160aac admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:06.848362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1296/2618] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:06 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:06.852158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b2944-1811-4679-b2c0-32bc1d422674 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:06.871505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1323/2619] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:06 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:28:06.875679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f680a48-ecf3-4a5a-9cb2-a1cdfa160aac admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:06.919372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b2944-1811-4679-b2c0-32bc1d422674 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:06.933341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:06.942644 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f680a48-ecf3-4a5a-9cb2-a1cdfa160aac admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:06.948072 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605978c-069b-49a9-a9d1-f26a404d5b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:06.948744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605978c-069b-49a9-a9d1-f26a404d5b0f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.949029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605978c-069b-49a9-a9d1-f26a404d5b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.949310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605978c-069b-49a9-a9d1-f26a404d5b0f admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.950051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605978c-069b-49a9-a9d1-f26a404d5b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:06.952835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605978c-069b-49a9-a9d1-f26a404d5b0f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:06.977604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1324/2620] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:06 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:28:06.978848 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605978c-069b-49a9-a9d1-f26a404d5b0f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:06.987138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b5c6d-c6f2-408f-915a-2780f4c6483d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:06.987914 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b5c6d-c6f2-408f-915a-2780f4c6483d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.988216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b5c6d-c6f2-408f-915a-2780f4c6483d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.988479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b5c6d-c6f2-408f-915a-2780f4c6483d admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.989154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b5c6d-c6f2-408f-915a-2780f4c6483d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:06.991064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605978c-069b-49a9-a9d1-f26a404d5b0f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:06.991210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b5c6d-c6f2-408f-915a-2780f4c6483d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:06.996893 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc5cd9-fa89-4170-9124-052952ab986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:06.997821 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc5cd9-fa89-4170-9124-052952ab986f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:06.998115 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc5cd9-fa89-4170-9124-052952ab986f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:06.998397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc5cd9-fa89-4170-9124-052952ab986f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:06.999132 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc5cd9-fa89-4170-9124-052952ab986f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.000725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc5cd9-fa89-4170-9124-052952ab986f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.013926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1325/2621] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:28:07.016802 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1297/2622] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:06 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:28:07.017915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc5cd9-fa89-4170-9124-052952ab986f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.023602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b5c6d-c6f2-408f-915a-2780f4c6483d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.029036 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc5cd9-fa89-4170-9124-052952ab986f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.034074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a9bff-3736-4dc1-92b9-a7228c531819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.034762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a9bff-3736-4dc1-92b9-a7228c531819 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.035031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a9bff-3736-4dc1-92b9-a7228c531819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.035312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a9bff-3736-4dc1-92b9-a7228c531819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.036299 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a9bff-3736-4dc1-92b9-a7228c531819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.037918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a9bff-3736-4dc1-92b9-a7228c531819 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.038030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b5c6d-c6f2-408f-915a-2780f4c6483d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.043542 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cf91c-4ed6-4e61-9b37-750e89a90195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.044223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cf91c-4ed6-4e61-9b37-750e89a90195 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.044489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cf91c-4ed6-4e61-9b37-750e89a90195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.044773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cf91c-4ed6-4e61-9b37-750e89a90195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.045489 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cf91c-4ed6-4e61-9b37-750e89a90195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.047110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cf91c-4ed6-4e61-9b37-750e89a90195 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.049981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1326/2623] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:28:07.054078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a9bff-3736-4dc1-92b9-a7228c531819 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.061904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1298/2624] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:07.062943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cf91c-4ed6-4e61-9b37-750e89a90195 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.068419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a9bff-3736-4dc1-92b9-a7228c531819 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.074880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421523fd-77f9-4a76-b777-1229982c296f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.075766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421523fd-77f9-4a76-b777-1229982c296f admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.076042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421523fd-77f9-4a76-b777-1229982c296f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.076313 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421523fd-77f9-4a76-b777-1229982c296f admin admin] PATH_INFO: `/v3/projects/09c74aae01ec47e5a49c27eb02e45333/users/2f3b6eba373c4c54945c7e98f8f2cc0f/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.076661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cf91c-4ed6-4e61-9b37-750e89a90195 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.082576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c27b7-c8d4-4bb6-a67e-d57ea75bc5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.083376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c27b7-c8d4-4bb6-a67e-d57ea75bc5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.083763 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c27b7-c8d4-4bb6-a67e-d57ea75bc5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.084594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c27b7-c8d4-4bb6-a67e-d57ea75bc5d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.085386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c27b7-c8d4-4bb6-a67e-d57ea75bc5d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.087364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c27b7-c8d4-4bb6-a67e-d57ea75bc5d6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.093588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421523fd-77f9-4a76-b777-1229982c296f admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c74aae01ec47e5a49c27eb02e45333, user_id=2f3b6eba373c4c54945c7e98f8f2cc0f, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.096322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421523fd-77f9-4a76-b777-1229982c296f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.101888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1299/2625] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:28:07.103841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c27b7-c8d4-4bb6-a67e-d57ea75bc5d6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.113180 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1327/2626] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:07 2022] PUT /identity/v3/projects/09c74aae01ec47e5a49c27eb02e45333/users/2f3b6eba373c4c54945c7e98f8f2cc0f/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:07.114126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c27b7-c8d4-4bb6-a67e-d57ea75bc5d6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.124899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbb290-31ff-4f24-b3f9-7d50d2086977 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.125198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbb290-31ff-4f24-b3f9-7d50d2086977 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.125474 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbb290-31ff-4f24-b3f9-7d50d2086977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.154574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c3dbb290-31ff-4f24-b3f9-7d50d2086977 None None] MFA Rules not processed for user `2f3b6eba373c4c54945c7e98f8f2cc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:07.155782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3dbb290-31ff-4f24-b3f9-7d50d2086977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:07.174851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bef126-e59e-4c84-aaae-1d27556cb088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.175744 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef126-e59e-4c84-aaae-1d27556cb088 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.176030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef126-e59e-4c84-aaae-1d27556cb088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.176306 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef126-e59e-4c84-aaae-1d27556cb088 admin admin] PATH_INFO: `/v3/projects/c239a2daefc145a4a3ce62d2b0c7d7c4/users/02a5cc12ab004ad2bcc1d3c1962bba12/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.191436 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bef126-e59e-4c84-aaae-1d27556cb088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c239a2daefc145a4a3ce62d2b0c7d7c4, user_id=02a5cc12ab004ad2bcc1d3c1962bba12, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.194275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bef126-e59e-4c84-aaae-1d27556cb088 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.208355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1300/2627] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:07 2022] PUT /identity/v3/projects/c239a2daefc145a4a3ce62d2b0c7d7c4/users/02a5cc12ab004ad2bcc1d3c1962bba12/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:07.214484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:07.218862 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218e8e8-3191-4803-ae02-3eec34bf359f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.219160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218e8e8-3191-4803-ae02-3eec34bf359f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.219437 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218e8e8-3191-4803-ae02-3eec34bf359f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.224863 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1328/2628] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:07.236090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbb290-31ff-4f24-b3f9-7d50d2086977 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.237511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3dbb290-31ff-4f24-b3f9-7d50d2086977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:07.246965 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4218e8e8-3191-4803-ae02-3eec34bf359f None None] MFA Rules not processed for user `02a5cc12ab004ad2bcc1d3c1962bba12`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:07.248153 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4218e8e8-3191-4803-ae02-3eec34bf359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:07.299327 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbb290-31ff-4f24-b3f9-7d50d2086977 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.310891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:07.324313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1301/2629] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:07.330822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218e8e8-3191-4803-ae02-3eec34bf359f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.357859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e4852-f08e-443b-8d7d-347041bb8559 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.358724 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e4852-f08e-443b-8d7d-347041bb8559 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.359066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e4852-f08e-443b-8d7d-347041bb8559 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.359402 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e4852-f08e-443b-8d7d-347041bb8559 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.404789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218e8e8-3191-4803-ae02-3eec34bf359f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.409693 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e93da0-4482-462b-8e7e-a9523be889b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.410468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e93da0-4482-462b-8e7e-a9523be889b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.410830 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e93da0-4482-462b-8e7e-a9523be889b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.411529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e93da0-4482-462b-8e7e-a9523be889b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.412290 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e93da0-4482-462b-8e7e-a9523be889b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.414340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e93da0-4482-462b-8e7e-a9523be889b2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.437767 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e4852-f08e-443b-8d7d-347041bb8559 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.439263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e4852-f08e-443b-8d7d-347041bb8559 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.450248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1329/2630] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:28:07.453598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1302/2631] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:07 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:07.457794 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e4852-f08e-443b-8d7d-347041bb8559 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.469441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e4852-f08e-443b-8d7d-347041bb8559 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.538593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922b211-3cd5-4921-a56b-bccf642c7a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.539168 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922b211-3cd5-4921-a56b-bccf642c7a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.539457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922b211-3cd5-4921-a56b-bccf642c7a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.539856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922b211-3cd5-4921-a56b-bccf642c7a2a admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.540408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922b211-3cd5-4921-a56b-bccf642c7a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.543586 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922b211-3cd5-4921-a56b-bccf642c7a2a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.553296 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:07.566222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1330/2632] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:07 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:07.569997 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e93da0-4482-462b-8e7e-a9523be889b2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.584979 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e93da0-4482-462b-8e7e-a9523be889b2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.592644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17ad1c-a776-4edc-b8a6-ce970ef024cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.593376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17ad1c-a776-4edc-b8a6-ce970ef024cc admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.593670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17ad1c-a776-4edc-b8a6-ce970ef024cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.594351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17ad1c-a776-4edc-b8a6-ce970ef024cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.595141 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17ad1c-a776-4edc-b8a6-ce970ef024cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.596739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17ad1c-a776-4edc-b8a6-ce970ef024cc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.611245 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1303/2633] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:07.615660 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922b211-3cd5-4921-a56b-bccf642c7a2a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.626755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922b211-3cd5-4921-a56b-bccf642c7a2a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.632693 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa0d8a-dfd8-4079-aefc-88fd7450b20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.633840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa0d8a-dfd8-4079-aefc-88fd7450b20b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.634157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa0d8a-dfd8-4079-aefc-88fd7450b20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.635461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa0d8a-dfd8-4079-aefc-88fd7450b20b admin admin] PATH_INFO: `/v3/projects/9a92b95f70dc4a0c800c60722e48eec0/users/59add34192b74916a022bca759b71aa7/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.657933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa0d8a-dfd8-4079-aefc-88fd7450b20b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a92b95f70dc4a0c800c60722e48eec0, user_id=59add34192b74916a022bca759b71aa7, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.660644 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa0d8a-dfd8-4079-aefc-88fd7450b20b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.675989 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1331/2634] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:07 2022] PUT /identity/v3/projects/9a92b95f70dc4a0c800c60722e48eec0/users/59add34192b74916a022bca759b71aa7/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:07.680435 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17ad1c-a776-4edc-b8a6-ce970ef024cc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.698418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17ad1c-a776-4edc-b8a6-ce970ef024cc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.784185 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bd068-7cf5-4606-84ea-874f2fb8148d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.784995 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bd068-7cf5-4606-84ea-874f2fb8148d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.785346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bd068-7cf5-4606-84ea-874f2fb8148d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.785686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bd068-7cf5-4606-84ea-874f2fb8148d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.786798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55bd068-7cf5-4606-84ea-874f2fb8148d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.788529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55bd068-7cf5-4606-84ea-874f2fb8148d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.806227 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1304/2635] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:07.807997 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa0d8a-dfd8-4079-aefc-88fd7450b20b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:07.823537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa0d8a-dfd8-4079-aefc-88fd7450b20b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:07.843969 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8aa7f-66dc-4847-a84e-a860972b92ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:07.844869 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8aa7f-66dc-4847-a84e-a860972b92ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.845051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8aa7f-66dc-4847-a84e-a860972b92ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.845523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8aa7f-66dc-4847-a84e-a860972b92ea admin admin] PATH_INFO: `/v3/projects/9a92b95f70dc4a0c800c60722e48eec0/users/59add34192b74916a022bca759b71aa7/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.848131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8aa7f-66dc-4847-a84e-a860972b92ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a92b95f70dc4a0c800c60722e48eec0, user_id=59add34192b74916a022bca759b71aa7, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:07.855699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8aa7f-66dc-4847-a84e-a860972b92ea admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:07.873688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1332/2636] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:07 2022] PUT /identity/v3/projects/9a92b95f70dc4a0c800c60722e48eec0/users/59add34192b74916a022bca759b71aa7/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:07.886039 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3fafa-b764-4832-8394-eb55ab757a88 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:07.886542 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3fafa-b764-4832-8394-eb55ab757a88 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:07.886826 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3fafa-b764-4832-8394-eb55ab757a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:07.922118 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2fd3fafa-b764-4832-8394-eb55ab757a88 None None] MFA Rules not processed for user `59add34192b74916a022bca759b71aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:07.923537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2fd3fafa-b764-4832-8394-eb55ab757a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:07.984908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:08.004991 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1305/2637] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:08.016145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8aa7f-66dc-4847-a84e-a860972b92ea admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:08.071666 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8aa7f-66dc-4847-a84e-a860972b92ea admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:08.075363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecebbed-79db-4e45-be08-7174b59a7e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:08.076001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecebbed-79db-4e45-be08-7174b59a7e6d service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:08.076287 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecebbed-79db-4e45-be08-7174b59a7e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:08.076565 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecebbed-79db-4e45-be08-7174b59a7e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:08.085484 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecebbed-79db-4e45-be08-7174b59a7e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:08.086952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecebbed-79db-4e45-be08-7174b59a7e6d service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:08.097391 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1333/2638] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:08.210517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3fafa-b764-4832-8394-eb55ab757a88 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:08.221337 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3fafa-b764-4832-8394-eb55ab757a88 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:08.225191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86eef7-eebd-49e2-a60b-8a06797e1037 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:08.226494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86eef7-eebd-49e2-a60b-8a06797e1037 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:08.226494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86eef7-eebd-49e2-a60b-8a06797e1037 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:08.226756 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86eef7-eebd-49e2-a60b-8a06797e1037 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:08.319275 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86eef7-eebd-49e2-a60b-8a06797e1037 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:08.323297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86eef7-eebd-49e2-a60b-8a06797e1037 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:08.333170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1306/2639] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:09.010681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7d59a-cd96-4432-8626-3aa61fd2a18d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:09.011151 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7d59a-cd96-4432-8626-3aa61fd2a18d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:09.011383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7d59a-cd96-4432-8626-3aa61fd2a18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:09.029888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d2a7d59a-cd96-4432-8626-3aa61fd2a18d None None] MFA Rules not processed for user `b36eda4f44b9428d9ff648979059fb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:09.031082 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d2a7d59a-cd96-4432-8626-3aa61fd2a18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:09.101205 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1334/2640] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:13.026963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86eef7-eebd-49e2-a60b-8a06797e1037 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:13.129246 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86eef7-eebd-49e2-a60b-8a06797e1037 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:13.136403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:13.138438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:13.139419 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:13.139830 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] PATH_INFO: `/v3/users/02a5cc12ab004ad2bcc1d3c1962bba12` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:13.143709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a5cc12ab004ad2bcc1d3c1962bba12)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:13.145966 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:13.228065 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a5cc12ab004ad2bcc1d3c1962bba12'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:13.237946 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a5cc12ab004ad2bcc1d3c1962bba12'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:13.238712 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a5cc12ab004ad2bcc1d3c1962bba12'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:13.255071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a5cc12ab004ad2bcc1d3c1962bba12'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:13.265544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a5cc12ab004ad2bcc1d3c1962bba12'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:13.274859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1307/2641] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:13 2022] DELETE /identity/v3/users/02a5cc12ab004ad2bcc1d3c1962bba12 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:13.279913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473c86f-1e42-458d-aaf4-c27951d2c11c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:13.280406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473c86f-1e42-458d-aaf4-c27951d2c11c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:13.280754 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473c86f-1e42-458d-aaf4-c27951d2c11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:13.305653 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2473c86f-1e42-458d-aaf4-c27951d2c11c None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:13.307470 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2473c86f-1e42-458d-aaf4-c27951d2c11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:13.378010 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1335/2642] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:13.387792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:13.389208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:13.403347 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:13.463865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9552d-03fa-4762-990d-910ba038863b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:13.468079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec380108-0a17-4893-a24f-c61c875a9b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:13.468676 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec380108-0a17-4893-a24f-c61c875a9b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:13.468959 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec380108-0a17-4893-a24f-c61c875a9b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:13.469234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec380108-0a17-4893-a24f-c61c875a9b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:13.489406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec380108-0a17-4893-a24f-c61c875a9b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:13.491057 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec380108-0a17-4893-a24f-c61c875a9b39 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:13.502325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1308/2643] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:13.824923 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473c86f-1e42-458d-aaf4-c27951d2c11c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:13.830729 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2473c86f-1e42-458d-aaf4-c27951d2c11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:13.872318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473c86f-1e42-458d-aaf4-c27951d2c11c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:13.880877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:13.883020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:13.883365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:13.883847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] PATH_INFO: `/v3/projects/c239a2daefc145a4a3ce62d2b0c7d7c4` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:13.885911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c239a2daefc145a4a3ce62d2b0c7d7c4)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:13.887907 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:13.890657 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:13.962149 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c239a2daefc145a4a3ce62d2b0c7d7c4', 'request_id': 'req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:13.977526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c239a2daefc145a4a3ce62d2b0c7d7c4', 'request_id': 'req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:13.986130 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c239a2daefc145a4a3ce62d2b0c7d7c4', 'request_id': 'req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:13.997643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] The token cache is being invalidate because project c239a2daefc145a4a3ce62d2b0c7d7c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:13.997966 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:13.998634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:14.000187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1336/2644] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:13 2022] DELETE /identity/v3/projects/c239a2daefc145a4a3ce62d2b0c7d7c4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:14.004444 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec380108-0a17-4893-a24f-c61c875a9b39 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:14.015016 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec380108-0a17-4893-a24f-c61c875a9b39 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:14.073929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e71b0-b35c-45e9-98b4-fb658a545db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:14.074869 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e71b0-b35c-45e9-98b4-fb658a545db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:14.075159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e71b0-b35c-45e9-98b4-fb658a545db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:14.075440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e71b0-b35c-45e9-98b4-fb658a545db9 admin admin] PATH_INFO: `/v3/users/59add34192b74916a022bca759b71aa7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:14.076643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e71b0-b35c-45e9-98b4-fb658a545db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59add34192b74916a022bca759b71aa7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:14.078607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e71b0-b35c-45e9-98b4-fb658a545db9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:14.126565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d7e71b0-b35c-45e9-98b4-fb658a545db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59add34192b74916a022bca759b71aa7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:14.136947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d7e71b0-b35c-45e9-98b4-fb658a545db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59add34192b74916a022bca759b71aa7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:14.137566 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d7e71b0-b35c-45e9-98b4-fb658a545db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59add34192b74916a022bca759b71aa7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:14.149437 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d7e71b0-b35c-45e9-98b4-fb658a545db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59add34192b74916a022bca759b71aa7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:14.156468 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d7e71b0-b35c-45e9-98b4-fb658a545db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59add34192b74916a022bca759b71aa7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:14.162683 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1309/2645] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:14 2022] DELETE /identity/v3/users/59add34192b74916a022bca759b71aa7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:14.890886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:14.892854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:14.915012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b08d9-2d79-4db1-8c15-aaa54fbb8a80 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:14.982323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:14.983215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:14.983496 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:14.983779 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] PATH_INFO: `/v3/projects/9a92b95f70dc4a0c800c60722e48eec0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:14.985113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a92b95f70dc4a0c800c60722e48eec0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:14.987061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:15.045750 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a92b95f70dc4a0c800c60722e48eec0', 'request_id': 'req-88330e5a-da65-49bb-9219-5a5d0ebcca09'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:15.052632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a92b95f70dc4a0c800c60722e48eec0', 'request_id': 'req-88330e5a-da65-49bb-9219-5a5d0ebcca09'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:15.066243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a92b95f70dc4a0c800c60722e48eec0', 'request_id': 'req-88330e5a-da65-49bb-9219-5a5d0ebcca09'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:15.079977 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] The token cache is being invalidate because project 9a92b95f70dc4a0c800c60722e48eec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:15.080413 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:15.081131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-88330e5a-da65-49bb-9219-5a5d0ebcca09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:15.083147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1337/2646] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:14 2022] DELETE /identity/v3/projects/9a92b95f70dc4a0c800c60722e48eec0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:15.088876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78708174-17dc-49d7-bb5e-798cf0eb6682 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.089362 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78708174-17dc-49d7-bb5e-798cf0eb6682 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.089564 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78708174-17dc-49d7-bb5e-798cf0eb6682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.115885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-78708174-17dc-49d7-bb5e-798cf0eb6682 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:15.115885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-78708174-17dc-49d7-bb5e-798cf0eb6682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:15.138873 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:15.176748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1310/2647] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:15.185553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcbe95-4930-4800-afd6-73f1a779ddd2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.185910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcbe95-4930-4800-afd6-73f1a779ddd2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.186190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcbe95-4930-4800-afd6-73f1a779ddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.204666 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-91fcbe95-4930-4800-afd6-73f1a779ddd2 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:15.205897 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-91fcbe95-4930-4800-afd6-73f1a779ddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:15.214479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1338/2648] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:15.218897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78708174-17dc-49d7-bb5e-798cf0eb6682 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.229236 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78708174-17dc-49d7-bb5e-798cf0eb6682 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:15.234667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618b5e0-4acd-4880-b17f-07f1e6b7b034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:15.235384 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618b5e0-4acd-4880-b17f-07f1e6b7b034 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.235722 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618b5e0-4acd-4880-b17f-07f1e6b7b034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.236000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618b5e0-4acd-4880-b17f-07f1e6b7b034 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.236734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618b5e0-4acd-4880-b17f-07f1e6b7b034 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:28:15.237089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618b5e0-4acd-4880-b17f-07f1e6b7b034 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:15.239036 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618b5e0-4acd-4880-b17f-07f1e6b7b034 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:15.255877 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1311/2649] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:28:15.256999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcbe95-4930-4800-afd6-73f1a779ddd2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.267306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcbe95-4930-4800-afd6-73f1a779ddd2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:15.281887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0f171-e7ac-4e03-9716-15e59bdd5ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:15.282853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0f171-e7ac-4e03-9716-15e59bdd5ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.283164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0f171-e7ac-4e03-9716-15e59bdd5ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.283732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0f171-e7ac-4e03-9716-15e59bdd5ed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.284896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0f171-e7ac-4e03-9716-15e59bdd5ed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:15.287219 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0f171-e7ac-4e03-9716-15e59bdd5ed4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:15.298659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:15.321747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1339/2650] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:15 2022] POST /identity/v3/projects => generated 407 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:28:15.326094 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618b5e0-4acd-4880-b17f-07f1e6b7b034 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.335970 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618b5e0-4acd-4880-b17f-07f1e6b7b034 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:15.393133 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40a128-3999-4684-983d-d0002611ff5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:15.393757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40a128-3999-4684-983d-d0002611ff5a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.394047 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40a128-3999-4684-983d-d0002611ff5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.394355 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40a128-3999-4684-983d-d0002611ff5a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.395034 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40a128-3999-4684-983d-d0002611ff5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:15.396954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40a128-3999-4684-983d-d0002611ff5a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:15.417613 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1312/2651] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:15 2022] POST /identity/v3/users => generated 304 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:15.422013 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0f171-e7ac-4e03-9716-15e59bdd5ed4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.432599 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0f171-e7ac-4e03-9716-15e59bdd5ed4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:15.439154 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e3d44-ead3-49d9-96da-de5aa98ff341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:15.439902 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e3d44-ead3-49d9-96da-de5aa98ff341 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.440397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e3d44-ead3-49d9-96da-de5aa98ff341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.440891 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e3d44-ead3-49d9-96da-de5aa98ff341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.441870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e3d44-ead3-49d9-96da-de5aa98ff341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:15.443481 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e3d44-ead3-49d9-96da-de5aa98ff341 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:15.454332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1340/2652] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:15.458476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40a128-3999-4684-983d-d0002611ff5a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.468972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40a128-3999-4684-983d-d0002611ff5a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:15.473819 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313d1cc-a0c6-4887-9cd6-2676a0d7ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:15.474483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313d1cc-a0c6-4887-9cd6-2676a0d7ff8f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.475077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313d1cc-a0c6-4887-9cd6-2676a0d7ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.475077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313d1cc-a0c6-4887-9cd6-2676a0d7ff8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.475771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313d1cc-a0c6-4887-9cd6-2676a0d7ff8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:15.477433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313d1cc-a0c6-4887-9cd6-2676a0d7ff8f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:15.490711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1313/2653] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:15.493541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e3d44-ead3-49d9-96da-de5aa98ff341 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.503914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e3d44-ead3-49d9-96da-de5aa98ff341 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:15.508915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d463de1-31c1-4e4b-8984-e217295537af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:15.510022 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d463de1-31c1-4e4b-8984-e217295537af admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.510460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d463de1-31c1-4e4b-8984-e217295537af admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.510739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d463de1-31c1-4e4b-8984-e217295537af admin admin] PATH_INFO: `/v3/projects/bb1c6795c1814b89a52cb4752d949ac8/users/750abab2c6ec4e5b91a15300e475cfb9/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.527918 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d463de1-31c1-4e4b-8984-e217295537af admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb1c6795c1814b89a52cb4752d949ac8, user_id=750abab2c6ec4e5b91a15300e475cfb9, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:15.530747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d463de1-31c1-4e4b-8984-e217295537af admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:15.546141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1341/2654] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:15 2022] PUT /identity/v3/projects/bb1c6795c1814b89a52cb4752d949ac8/users/750abab2c6ec4e5b91a15300e475cfb9/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:15.557322 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56276e4-9e17-4c4c-8fd6-0961b730364d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.557615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56276e4-9e17-4c4c-8fd6-0961b730364d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.558488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56276e4-9e17-4c4c-8fd6-0961b730364d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.585372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d56276e4-9e17-4c4c-8fd6-0961b730364d None None] MFA Rules not processed for user `750abab2c6ec4e5b91a15300e475cfb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:15.588108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d56276e4-9e17-4c4c-8fd6-0961b730364d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:15.659413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1314/2655] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:15.663934 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d463de1-31c1-4e4b-8984-e217295537af admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.738213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d463de1-31c1-4e4b-8984-e217295537af admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:15.743336 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4452a7-ba65-4a5f-8627-1004aa78f3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:15.744077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4452a7-ba65-4a5f-8627-1004aa78f3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.744442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4452a7-ba65-4a5f-8627-1004aa78f3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.744635 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4452a7-ba65-4a5f-8627-1004aa78f3d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.745324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4452a7-ba65-4a5f-8627-1004aa78f3d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:15.748298 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4452a7-ba65-4a5f-8627-1004aa78f3d2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:15.780304 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1342/2656] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:15 2022] POST /identity/v3/projects => generated 409 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:15.781514 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56276e4-9e17-4c4c-8fd6-0961b730364d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.794325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56276e4-9e17-4c4c-8fd6-0961b730364d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:15.819445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:15.854304 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54626f35-6f81-4216-a313-bd43e01a5438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:15.855114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54626f35-6f81-4216-a313-bd43e01a5438 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.855400 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54626f35-6f81-4216-a313-bd43e01a5438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.855677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54626f35-6f81-4216-a313-bd43e01a5438 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.856370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54626f35-6f81-4216-a313-bd43e01a5438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:15.858356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54626f35-6f81-4216-a313-bd43e01a5438 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:15.880711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1315/2657] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:15 2022] POST /identity/v3/users => generated 311 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:15.882061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4452a7-ba65-4a5f-8627-1004aa78f3d2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.894572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4452a7-ba65-4a5f-8627-1004aa78f3d2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:15.899561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0f809-c355-475e-b913-1c4b181f6857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:15.900911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0f809-c355-475e-b913-1c4b181f6857 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.901193 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0f809-c355-475e-b913-1c4b181f6857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.901473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0f809-c355-475e-b913-1c4b181f6857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.902271 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0f809-c355-475e-b913-1c4b181f6857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:15.904072 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0f809-c355-475e-b913-1c4b181f6857 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:15.915100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54626f35-6f81-4216-a313-bd43e01a5438 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.918360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1343/2658] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:15.918497 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-54626f35-6f81-4216-a313-bd43e01a5438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:15.919376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0f809-c355-475e-b913-1c4b181f6857 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.929276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0f809-c355-475e-b913-1c4b181f6857 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:15.934216 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82078e4c-37c8-4db0-8db6-6fb15aeb1f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:15.935084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82078e4c-37c8-4db0-8db6-6fb15aeb1f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:15.935370 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82078e4c-37c8-4db0-8db6-6fb15aeb1f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:15.935645 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82078e4c-37c8-4db0-8db6-6fb15aeb1f94 admin admin] PATH_INFO: `/v3/projects/c95e7b8b6b0f4942a4c5828871fdf1ab/users/32bedc5db2ac418495e9c25600227d89/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:15.950554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82078e4c-37c8-4db0-8db6-6fb15aeb1f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95e7b8b6b0f4942a4c5828871fdf1ab, user_id=32bedc5db2ac418495e9c25600227d89, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:15.953259 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82078e4c-37c8-4db0-8db6-6fb15aeb1f94 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:15.968310 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1344/2659] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:15 2022] PUT /identity/v3/projects/c95e7b8b6b0f4942a4c5828871fdf1ab/users/32bedc5db2ac418495e9c25600227d89/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:15.972324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82078e4c-37c8-4db0-8db6-6fb15aeb1f94 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:15.977353 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54626f35-6f81-4216-a313-bd43e01a5438 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:16.027708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3608af-0e43-47b2-83d5-07176ae7aecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:16.028360 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3608af-0e43-47b2-83d5-07176ae7aecf service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:16.028644 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3608af-0e43-47b2-83d5-07176ae7aecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:16.028975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3608af-0e43-47b2-83d5-07176ae7aecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:16.030933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9c3608af-0e43-47b2-83d5-07176ae7aecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:16.033940 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82078e4c-37c8-4db0-8db6-6fb15aeb1f94 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:16.039886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d054d92-94cf-4188-afcb-2c4faa504611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:16.040820 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d054d92-94cf-4188-afcb-2c4faa504611 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:16.041109 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d054d92-94cf-4188-afcb-2c4faa504611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:16.041397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d054d92-94cf-4188-afcb-2c4faa504611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:16.042321 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d054d92-94cf-4188-afcb-2c4faa504611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:16.043975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d054d92-94cf-4188-afcb-2c4faa504611 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:16.053892 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1345/2660] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:28:16.058276 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d054d92-94cf-4188-afcb-2c4faa504611 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:16.063383 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:16.069490 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d054d92-94cf-4188-afcb-2c4faa504611 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:16.075316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b80b44-3203-40c6-afb5-e6ab5f88380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:16.076184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b80b44-3203-40c6-afb5-e6ab5f88380f admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:16.076461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b80b44-3203-40c6-afb5-e6ab5f88380f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:16.076733 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b80b44-3203-40c6-afb5-e6ab5f88380f admin admin] PATH_INFO: `/v3/projects/c95e7b8b6b0f4942a4c5828871fdf1ab/users/32bedc5db2ac418495e9c25600227d89/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:16.078771 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b80b44-3203-40c6-afb5-e6ab5f88380f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95e7b8b6b0f4942a4c5828871fdf1ab, user_id=32bedc5db2ac418495e9c25600227d89, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:16.081389 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b80b44-3203-40c6-afb5-e6ab5f88380f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:16.092982 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1346/2661] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:16 2022] PUT /identity/v3/projects/c95e7b8b6b0f4942a4c5828871fdf1ab/users/32bedc5db2ac418495e9c25600227d89/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:16.096601 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3608af-0e43-47b2-83d5-07176ae7aecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:16.098154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3608af-0e43-47b2-83d5-07176ae7aecf service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:16.104090 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98b59-2d36-4264-b51a-a3ea89ef0ff5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:16.104385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98b59-2d36-4264-b51a-a3ea89ef0ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:16.104661 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98b59-2d36-4264-b51a-a3ea89ef0ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:16.108616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1316/2662] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:16.135845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cfc98b59-2d36-4264-b51a-a3ea89ef0ff5 None None] MFA Rules not processed for user `32bedc5db2ac418495e9c25600227d89`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:16.137027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cfc98b59-2d36-4264-b51a-a3ea89ef0ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:16.144121 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3608af-0e43-47b2-83d5-07176ae7aecf service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:16.145804 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9c3608af-0e43-47b2-83d5-07176ae7aecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:16.160553 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3608af-0e43-47b2-83d5-07176ae7aecf service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:16.181042 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:16.220603 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1347/2663] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:16 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:16.231932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc98b59-2d36-4264-b51a-a3ea89ef0ff5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:16.232925 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752d192-a638-4197-a75b-2e5f795dec98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:16.233255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cfc98b59-2d36-4264-b51a-a3ea89ef0ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:16.234212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752d192-a638-4197-a75b-2e5f795dec98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:16.234598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752d192-a638-4197-a75b-2e5f795dec98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:16.235503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752d192-a638-4197-a75b-2e5f795dec98 admin admin] PATH_INFO: `/v3/users/2f3b6eba373c4c54945c7e98f8f2cc0f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:16.236361 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752d192-a638-4197-a75b-2e5f795dec98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f3b6eba373c4c54945c7e98f8f2cc0f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:16.238641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752d192-a638-4197-a75b-2e5f795dec98 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:16.300222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c752d192-a638-4197-a75b-2e5f795dec98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f3b6eba373c4c54945c7e98f8f2cc0f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:16.311722 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc98b59-2d36-4264-b51a-a3ea89ef0ff5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:16.312884 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c752d192-a638-4197-a75b-2e5f795dec98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f3b6eba373c4c54945c7e98f8f2cc0f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:16.314001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c752d192-a638-4197-a75b-2e5f795dec98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f3b6eba373c4c54945c7e98f8f2cc0f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:16.322832 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa6e63-bb60-4766-a451-aca0bce88842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:16.323694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa6e63-bb60-4766-a451-aca0bce88842 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:16.323980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa6e63-bb60-4766-a451-aca0bce88842 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:16.324258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa6e63-bb60-4766-a451-aca0bce88842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:16.326744 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-84aa6e63-bb60-4766-a451-aca0bce88842 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:16.334424 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c752d192-a638-4197-a75b-2e5f795dec98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f3b6eba373c4c54945c7e98f8f2cc0f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:16.344406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa6e63-bb60-4766-a451-aca0bce88842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:16.346472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa6e63-bb60-4766-a451-aca0bce88842 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:16.346604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c752d192-a638-4197-a75b-2e5f795dec98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f3b6eba373c4c54945c7e98f8f2cc0f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:16.355134 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1317/2664] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:16 2022] DELETE /identity/v3/users/2f3b6eba373c4c54945c7e98f8f2cc0f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:16.359175 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1348/2665] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:16 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:28:16.359394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f489c-3d1a-4059-88ae-7ff72ccdf4d5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:16.359697 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f489c-3d1a-4059-88ae-7ff72ccdf4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:16.359981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f489c-3d1a-4059-88ae-7ff72ccdf4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:16.380039 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b18f489c-3d1a-4059-88ae-7ff72ccdf4d5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:16.381408 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b18f489c-3d1a-4059-88ae-7ff72ccdf4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:16.484924 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1318/2666] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:16.495574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa6e63-bb60-4766-a451-aca0bce88842 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:16.497925 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-84aa6e63-bb60-4766-a451-aca0bce88842 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:16.582849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f489c-3d1a-4059-88ae-7ff72ccdf4d5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:16.584458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b18f489c-3d1a-4059-88ae-7ff72ccdf4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:16.585381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa6e63-bb60-4766-a451-aca0bce88842 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:16.622731 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:16.638861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:16.654524 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661e409-d23b-4981-b0e0-b03d26521052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:16.655654 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661e409-d23b-4981-b0e0-b03d26521052 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:16.656009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661e409-d23b-4981-b0e0-b03d26521052 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:16.656351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661e409-d23b-4981-b0e0-b03d26521052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:16.659699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f489c-3d1a-4059-88ae-7ff72ccdf4d5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:16.663986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fa39f-2938-4f88-a56d-0582e75033f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:16.664608 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fa39f-2938-4f88-a56d-0582e75033f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:16.664940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fa39f-2938-4f88-a56d-0582e75033f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:16.665164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fa39f-2938-4f88-a56d-0582e75033f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:16.667298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-134fa39f-2938-4f88-a56d-0582e75033f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:16.670197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661e409-d23b-4981-b0e0-b03d26521052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:16.671797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661e409-d23b-4981-b0e0-b03d26521052 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:16.693846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1349/2667] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:16.759017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fa39f-2938-4f88-a56d-0582e75033f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:16.770807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fa39f-2938-4f88-a56d-0582e75033f1 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:16.796735 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1319/2668] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:16 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:28:17.132692 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661e409-d23b-4981-b0e0-b03d26521052 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:17.135172 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7661e409-d23b-4981-b0e0-b03d26521052 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:17.173416 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661e409-d23b-4981-b0e0-b03d26521052 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:17.178392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:17.179465 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:17.179745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:17.181659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] PATH_INFO: `/v3/projects/09c74aae01ec47e5a49c27eb02e45333` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:17.182058 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c74aae01ec47e5a49c27eb02e45333)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:17.184237 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:17.236202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c74aae01ec47e5a49c27eb02e45333', 'request_id': 'req-58efc658-5ef3-47dc-81be-5527f54a8014'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:17.244391 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c74aae01ec47e5a49c27eb02e45333', 'request_id': 'req-58efc658-5ef3-47dc-81be-5527f54a8014'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:17.253027 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c74aae01ec47e5a49c27eb02e45333', 'request_id': 'req-58efc658-5ef3-47dc-81be-5527f54a8014'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:17.273375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] The token cache is being invalidate because project 09c74aae01ec47e5a49c27eb02e45333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:17.273936 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:17.275283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58efc658-5ef3-47dc-81be-5527f54a8014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:17.278138 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1350/2669] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:17 2022] DELETE /identity/v3/projects/09c74aae01ec47e5a49c27eb02e45333 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:17.285654 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d32c2-6113-4426-a3b5-0ddad3ce0e80 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:17.286432 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d32c2-6113-4426-a3b5-0ddad3ce0e80 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:17.286722 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d32c2-6113-4426-a3b5-0ddad3ce0e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:17.305865 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-784d32c2-6113-4426-a3b5-0ddad3ce0e80 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:17.308100 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-784d32c2-6113-4426-a3b5-0ddad3ce0e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:17.369617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1320/2670] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:17.377167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b97e9b-540a-47d7-9dab-dff5394f6784 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:17.377533 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b97e9b-540a-47d7-9dab-dff5394f6784 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:17.377910 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b97e9b-540a-47d7-9dab-dff5394f6784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:17.396220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-83b97e9b-540a-47d7-9dab-dff5394f6784 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:17.397451 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-83b97e9b-540a-47d7-9dab-dff5394f6784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:17.406301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1351/2671] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:17.410350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d32c2-6113-4426-a3b5-0ddad3ce0e80 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:17.421256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d32c2-6113-4426-a3b5-0ddad3ce0e80 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:17.426270 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ec67e-3529-44d5-9659-034d6804ac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:17.426942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec67e-3529-44d5-9659-034d6804ac13 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:17.427255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec67e-3529-44d5-9659-034d6804ac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:17.427516 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec67e-3529-44d5-9659-034d6804ac13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:17.428262 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ec67e-3529-44d5-9659-034d6804ac13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:28:17.429327 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ec67e-3529-44d5-9659-034d6804ac13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:17.431032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ec67e-3529-44d5-9659-034d6804ac13 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:17.446915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1321/2672] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:28:17.451231 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b97e9b-540a-47d7-9dab-dff5394f6784 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:17.461434 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b97e9b-540a-47d7-9dab-dff5394f6784 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:17.467590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ca1a4-c905-46c5-b979-d47b3f426461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:17.468786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ca1a4-c905-46c5-b979-d47b3f426461 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:17.469354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ca1a4-c905-46c5-b979-d47b3f426461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:17.469681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ca1a4-c905-46c5-b979-d47b3f426461 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:17.470834 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ca1a4-c905-46c5-b979-d47b3f426461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:17.473256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ca1a4-c905-46c5-b979-d47b3f426461 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:17.505024 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1352/2673] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:17 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:17.506382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ec67e-3529-44d5-9659-034d6804ac13 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:17.516900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ec67e-3529-44d5-9659-034d6804ac13 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:17.576895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bffb7-59bb-44c2-b16e-665b9dea5b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:17.577569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bffb7-59bb-44c2-b16e-665b9dea5b15 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:17.577880 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bffb7-59bb-44c2-b16e-665b9dea5b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:17.578148 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bffb7-59bb-44c2-b16e-665b9dea5b15 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:17.578822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1bffb7-59bb-44c2-b16e-665b9dea5b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:17.580728 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1bffb7-59bb-44c2-b16e-665b9dea5b15 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:17.602674 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1322/2674] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:17 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:28:17.607466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ca1a4-c905-46c5-b979-d47b3f426461 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:17.617556 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ca1a4-c905-46c5-b979-d47b3f426461 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:17.623183 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55255e9-a447-4e4b-98a8-f0fc28a702f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:17.623853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55255e9-a447-4e4b-98a8-f0fc28a702f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:17.624148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55255e9-a447-4e4b-98a8-f0fc28a702f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:17.624429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55255e9-a447-4e4b-98a8-f0fc28a702f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:17.625196 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55255e9-a447-4e4b-98a8-f0fc28a702f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:17.626904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55255e9-a447-4e4b-98a8-f0fc28a702f5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:17.640409 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1353/2675] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:17.641586 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bffb7-59bb-44c2-b16e-665b9dea5b15 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:17.652711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bffb7-59bb-44c2-b16e-665b9dea5b15 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:17.659531 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28bb8b-2881-44dc-b83a-ff3121196fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:17.660195 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28bb8b-2881-44dc-b83a-ff3121196fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:17.660486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28bb8b-2881-44dc-b83a-ff3121196fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:17.660762 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28bb8b-2881-44dc-b83a-ff3121196fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:17.661845 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28bb8b-2881-44dc-b83a-ff3121196fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:17.663709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28bb8b-2881-44dc-b83a-ff3121196fb8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:17.674757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1323/2676] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:17.678804 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55255e9-a447-4e4b-98a8-f0fc28a702f5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:17.689643 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55255e9-a447-4e4b-98a8-f0fc28a702f5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:17.694949 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca943f-0d18-49d4-a7c8-dfc2ebbe8e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:17.695842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca943f-0d18-49d4-a7c8-dfc2ebbe8e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:17.696171 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca943f-0d18-49d4-a7c8-dfc2ebbe8e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:17.696790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca943f-0d18-49d4-a7c8-dfc2ebbe8e54 admin admin] PATH_INFO: `/v3/projects/f65875f3e7b640b799d1ddf5c88e212d/users/2aeadda12d6546449852095fb85e072b/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:17.712945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca943f-0d18-49d4-a7c8-dfc2ebbe8e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f65875f3e7b640b799d1ddf5c88e212d, user_id=2aeadda12d6546449852095fb85e072b, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:17.715806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca943f-0d18-49d4-a7c8-dfc2ebbe8e54 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:17.729466 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1354/2677] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:17 2022] PUT /identity/v3/projects/f65875f3e7b640b799d1ddf5c88e212d/users/2aeadda12d6546449852095fb85e072b/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:17.740602 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ffa1-3c15-4071-bcff-0f292382703b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:17.740972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ffa1-3c15-4071-bcff-0f292382703b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:17.741334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ffa1-3c15-4071-bcff-0f292382703b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:17.747845 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:17.776055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-26f1ffa1-3c15-4071-bcff-0f292382703b None None] MFA Rules not processed for user `2aeadda12d6546449852095fb85e072b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:17.776055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-26f1ffa1-3c15-4071-bcff-0f292382703b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:17.849585 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1324/2678] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:17.863045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca943f-0d18-49d4-a7c8-dfc2ebbe8e54 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:17.864825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1fca943f-0d18-49d4-a7c8-dfc2ebbe8e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:17.943190 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca943f-0d18-49d4-a7c8-dfc2ebbe8e54 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:17.947113 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4f735-39a8-4eb2-8fcc-68d1eb41d4f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8386ca93fe48460ba5b6f0c10dc51122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:17.947974 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4f735-39a8-4eb2-8fcc-68d1eb41d4f5 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:17.948264 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4f735-39a8-4eb2-8fcc-68d1eb41d4f5 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:17.948548 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4f735-39a8-4eb2-8fcc-68d1eb41d4f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:17.958019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4f735-39a8-4eb2-8fcc-68d1eb41d4f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:17.959525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4f735-39a8-4eb2-8fcc-68d1eb41d4f5 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:17.965311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:17.972714 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1355/2679] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:24.402648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1ffa1-3c15-4071-bcff-0f292382703b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:24.404131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-26f1ffa1-3c15-4071-bcff-0f292382703b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:24.476973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1ffa1-3c15-4071-bcff-0f292382703b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:24.482375 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:24.483289 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:24.483572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:24.483905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] PATH_INFO: `/v3/users/750abab2c6ec4e5b91a15300e475cfb9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:24.485096 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=750abab2c6ec4e5b91a15300e475cfb9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:24.487119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:24.534442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '750abab2c6ec4e5b91a15300e475cfb9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:24.546685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '750abab2c6ec4e5b91a15300e475cfb9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:24.547312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '750abab2c6ec4e5b91a15300e475cfb9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:24.562179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '750abab2c6ec4e5b91a15300e475cfb9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:24.569143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '750abab2c6ec4e5b91a15300e475cfb9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:24.575572 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1325/2680] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:24 2022] DELETE /identity/v3/users/750abab2c6ec4e5b91a15300e475cfb9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:24.580207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df1fcb-96ee-4bd6-9122-32b5ce44ed50 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:24.580718 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df1fcb-96ee-4bd6-9122-32b5ce44ed50 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:24.580844 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df1fcb-96ee-4bd6-9122-32b5ce44ed50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:24.599394 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-57df1fcb-96ee-4bd6-9122-32b5ce44ed50 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:24.600605 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-57df1fcb-96ee-4bd6-9122-32b5ce44ed50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:24.661952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1356/2681] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:24.672885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:24.674949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:24.691021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:24.741405 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04f40d-bbda-443d-aa4b-0e4159a93c22 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:24.745516 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba8d30-74a5-413e-be7c-80fa5a20df1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:24.746506 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba8d30-74a5-413e-be7c-80fa5a20df1f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:24.746792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba8d30-74a5-413e-be7c-80fa5a20df1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:24.747574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba8d30-74a5-413e-be7c-80fa5a20df1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:24.761402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba8d30-74a5-413e-be7c-80fa5a20df1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:24.762929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba8d30-74a5-413e-be7c-80fa5a20df1f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:24.772747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1326/2682] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:25.083870 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df1fcb-96ee-4bd6-9122-32b5ce44ed50 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:25.084913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-57df1fcb-96ee-4bd6-9122-32b5ce44ed50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:25.098037 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df1fcb-96ee-4bd6-9122-32b5ce44ed50 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:25.103118 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:25.103996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:25.104274 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:25.104546 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] PATH_INFO: `/v3/projects/bb1c6795c1814b89a52cb4752d949ac8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:25.105757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb1c6795c1814b89a52cb4752d949ac8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:25.107691 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:25.160659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb1c6795c1814b89a52cb4752d949ac8', 'request_id': 'req-745738f1-ed42-4daf-a56a-b35afbfa1472'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.167526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb1c6795c1814b89a52cb4752d949ac8', 'request_id': 'req-745738f1-ed42-4daf-a56a-b35afbfa1472'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.175035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb1c6795c1814b89a52cb4752d949ac8', 'request_id': 'req-745738f1-ed42-4daf-a56a-b35afbfa1472'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.186084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] The token cache is being invalidate because project bb1c6795c1814b89a52cb4752d949ac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:25.186380 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.187012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.188950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1357/2683] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:25 2022] DELETE /identity/v3/projects/bb1c6795c1814b89a52cb4752d949ac8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:25.193230 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba8d30-74a5-413e-be7c-80fa5a20df1f service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:25.203949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba8d30-74a5-413e-be7c-80fa5a20df1f service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:25.266519 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dbfcb-3d3c-4417-942e-253481c23927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:25.267503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dbfcb-3d3c-4417-942e-253481c23927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:25.267871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dbfcb-3d3c-4417-942e-253481c23927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:25.268179 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dbfcb-3d3c-4417-942e-253481c23927 admin admin] PATH_INFO: `/v3/users/32bedc5db2ac418495e9c25600227d89` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:25.269521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dbfcb-3d3c-4417-942e-253481c23927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32bedc5db2ac418495e9c25600227d89)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:25.271511 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dbfcb-3d3c-4417-942e-253481c23927 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:25.323621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b6dbfcb-3d3c-4417-942e-253481c23927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32bedc5db2ac418495e9c25600227d89'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.331344 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b6dbfcb-3d3c-4417-942e-253481c23927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32bedc5db2ac418495e9c25600227d89'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.331973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b6dbfcb-3d3c-4417-942e-253481c23927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32bedc5db2ac418495e9c25600227d89'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.343905 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b6dbfcb-3d3c-4417-942e-253481c23927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32bedc5db2ac418495e9c25600227d89'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.351403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b6dbfcb-3d3c-4417-942e-253481c23927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32bedc5db2ac418495e9c25600227d89'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.358328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1327/2684] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:25 2022] DELETE /identity/v3/users/32bedc5db2ac418495e9c25600227d89 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:25.636023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:25.637492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:25.650730 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745738f1-ed42-4daf-a56a-b35afbfa1472 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:25.712142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:25.713008 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:25.713293 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:25.713567 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] PATH_INFO: `/v3/projects/c95e7b8b6b0f4942a4c5828871fdf1ab` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:25.714879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95e7b8b6b0f4942a4c5828871fdf1ab)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:25.716987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:25.742341 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:25.767014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95e7b8b6b0f4942a4c5828871fdf1ab', 'request_id': 'req-afb37601-b9af-4e83-bdaf-d16c2747281f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.773128 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95e7b8b6b0f4942a4c5828871fdf1ab', 'request_id': 'req-afb37601-b9af-4e83-bdaf-d16c2747281f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.780971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95e7b8b6b0f4942a4c5828871fdf1ab', 'request_id': 'req-afb37601-b9af-4e83-bdaf-d16c2747281f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.792110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] The token cache is being invalidate because project c95e7b8b6b0f4942a4c5828871fdf1ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:25.792399 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.793016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-afb37601-b9af-4e83-bdaf-d16c2747281f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:25.794584 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1358/2685] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:25 2022] DELETE /identity/v3/projects/c95e7b8b6b0f4942a4c5828871fdf1ab => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:25.799394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f357-d806-4553-91bc-75be4d0a57df None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:25.799876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f357-d806-4553-91bc-75be4d0a57df None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:25.799981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f357-d806-4553-91bc-75be4d0a57df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:25.819496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-aa02f357-d806-4553-91bc-75be4d0a57df None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:25.820741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aa02f357-d806-4553-91bc-75be4d0a57df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:25.850888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:25.887049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1328/2686] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:25.894880 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cb1e1-2afd-48a7-aaa7-25697dbd1102 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:25.895238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cb1e1-2afd-48a7-aaa7-25697dbd1102 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:25.895495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cb1e1-2afd-48a7-aaa7-25697dbd1102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:25.916611 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c31cb1e1-2afd-48a7-aaa7-25697dbd1102 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:25.917866 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c31cb1e1-2afd-48a7-aaa7-25697dbd1102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:25.925884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1359/2687] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:25.930105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f357-d806-4553-91bc-75be4d0a57df None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:25.939891 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f357-d806-4553-91bc-75be4d0a57df None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:25.945105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375cd42-ad6e-4e75-8fff-f27d15683fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:25.945920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375cd42-ad6e-4e75-8fff-f27d15683fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:25.946295 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375cd42-ad6e-4e75-8fff-f27d15683fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:25.946579 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375cd42-ad6e-4e75-8fff-f27d15683fb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:25.947492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375cd42-ad6e-4e75-8fff-f27d15683fb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:28:25.947813 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375cd42-ad6e-4e75-8fff-f27d15683fb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:25.949543 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375cd42-ad6e-4e75-8fff-f27d15683fb8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:25.961613 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1329/2688] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:28:25.966517 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cb1e1-2afd-48a7-aaa7-25697dbd1102 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:25.975574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cb1e1-2afd-48a7-aaa7-25697dbd1102 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:25.980976 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1cf7c-87b0-4246-83a2-031d32c47899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:25.981623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1cf7c-87b0-4246-83a2-031d32c47899 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:25.981945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1cf7c-87b0-4246-83a2-031d32c47899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:25.982223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1cf7c-87b0-4246-83a2-031d32c47899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:25.982904 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1cf7c-87b0-4246-83a2-031d32c47899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:25.984799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1cf7c-87b0-4246-83a2-031d32c47899 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:26.011112 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1360/2689] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:25 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:26.015459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375cd42-ad6e-4e75-8fff-f27d15683fb8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:26.026208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375cd42-ad6e-4e75-8fff-f27d15683fb8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:26.088795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa5921-90eb-4cc2-9fe1-6a361c38309e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:26.089688 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa5921-90eb-4cc2-9fe1-6a361c38309e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.090017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa5921-90eb-4cc2-9fe1-6a361c38309e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.090298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa5921-90eb-4cc2-9fe1-6a361c38309e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.091980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa5921-90eb-4cc2-9fe1-6a361c38309e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:26.092929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa5921-90eb-4cc2-9fe1-6a361c38309e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:26.116231 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1330/2690] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:26 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:26.120412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1cf7c-87b0-4246-83a2-031d32c47899 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:26.131162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1cf7c-87b0-4246-83a2-031d32c47899 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:26.136176 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ac07e-dc49-4850-9bc6-563e890f0111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:26.136950 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ac07e-dc49-4850-9bc6-563e890f0111 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.137308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ac07e-dc49-4850-9bc6-563e890f0111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.137651 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ac07e-dc49-4850-9bc6-563e890f0111 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.138480 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ac07e-dc49-4850-9bc6-563e890f0111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:26.140152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ac07e-dc49-4850-9bc6-563e890f0111 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:26.152392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1361/2691] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:26.156377 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa5921-90eb-4cc2-9fe1-6a361c38309e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:26.172356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa5921-90eb-4cc2-9fe1-6a361c38309e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:26.177542 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60a972-4479-4295-8d22-7472694f6275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:26.178458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a972-4479-4295-8d22-7472694f6275 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.178750 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a972-4479-4295-8d22-7472694f6275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.179097 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a972-4479-4295-8d22-7472694f6275 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.179920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60a972-4479-4295-8d22-7472694f6275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:26.181563 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60a972-4479-4295-8d22-7472694f6275 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:26.192331 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1331/2692] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:26.196158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ac07e-dc49-4850-9bc6-563e890f0111 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:26.207699 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ac07e-dc49-4850-9bc6-563e890f0111 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:26.213493 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcddeb-97bc-4b9f-aaf2-3efe5d5832cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:26.214987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcddeb-97bc-4b9f-aaf2-3efe5d5832cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.215272 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcddeb-97bc-4b9f-aaf2-3efe5d5832cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.215557 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcddeb-97bc-4b9f-aaf2-3efe5d5832cf admin admin] PATH_INFO: `/v3/projects/0a829404dbe14d25b69f5a453fed1fb6/users/70d7248179af4af6b76e96ee46c9a94c/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.231707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcddeb-97bc-4b9f-aaf2-3efe5d5832cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a829404dbe14d25b69f5a453fed1fb6, user_id=70d7248179af4af6b76e96ee46c9a94c, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:26.234568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcddeb-97bc-4b9f-aaf2-3efe5d5832cf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:26.250563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1362/2693] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:26 2022] PUT /identity/v3/projects/0a829404dbe14d25b69f5a453fed1fb6/users/70d7248179af4af6b76e96ee46c9a94c/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:26.261462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff9980-97fe-4395-9e80-0eaaa06a6230 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.261825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff9980-97fe-4395-9e80-0eaaa06a6230 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.262111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff9980-97fe-4395-9e80-0eaaa06a6230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.293652 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-74ff9980-97fe-4395-9e80-0eaaa06a6230 None None] MFA Rules not processed for user `70d7248179af4af6b76e96ee46c9a94c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:26.295035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-74ff9980-97fe-4395-9e80-0eaaa06a6230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:26.371163 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1332/2694] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:26.375947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcddeb-97bc-4b9f-aaf2-3efe5d5832cf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:26.443921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcddeb-97bc-4b9f-aaf2-3efe5d5832cf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:26.454881 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fecdf-2f8d-4495-83e1-19710ed756c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:26.455573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fecdf-2f8d-4495-83e1-19710ed756c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.455859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fecdf-2f8d-4495-83e1-19710ed756c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.456177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fecdf-2f8d-4495-83e1-19710ed756c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.456887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fecdf-2f8d-4495-83e1-19710ed756c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:26.458889 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fecdf-2f8d-4495-83e1-19710ed756c3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:26.490473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1363/2695] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:26 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:26.494502 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff9980-97fe-4395-9e80-0eaaa06a6230 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:26.506310 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff9980-97fe-4395-9e80-0eaaa06a6230 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:26.535734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:26.569525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3008b-5d29-41e7-90bc-8815cba0ae4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:26.570558 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3008b-5d29-41e7-90bc-8815cba0ae4a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.570708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3008b-5d29-41e7-90bc-8815cba0ae4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.570980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3008b-5d29-41e7-90bc-8815cba0ae4a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.572300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3008b-5d29-41e7-90bc-8815cba0ae4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:26.574630 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3008b-5d29-41e7-90bc-8815cba0ae4a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:26.600690 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1333/2696] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:26 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:26.601919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fecdf-2f8d-4495-83e1-19710ed756c3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:26.612886 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fecdf-2f8d-4495-83e1-19710ed756c3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:26.618035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61964126-dc84-44f3-9fb0-a7eae05c525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:26.618797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61964126-dc84-44f3-9fb0-a7eae05c525c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.619108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61964126-dc84-44f3-9fb0-a7eae05c525c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.619433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61964126-dc84-44f3-9fb0-a7eae05c525c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.620163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61964126-dc84-44f3-9fb0-a7eae05c525c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:26.621990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61964126-dc84-44f3-9fb0-a7eae05c525c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:26.637511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1364/2697] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:26.639568 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3008b-5d29-41e7-90bc-8815cba0ae4a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:26.650190 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3008b-5d29-41e7-90bc-8815cba0ae4a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:26.654958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c394d05-51bd-46d7-8545-08f2671a0e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:26.655625 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c394d05-51bd-46d7-8545-08f2671a0e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.655915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c394d05-51bd-46d7-8545-08f2671a0e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.656191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c394d05-51bd-46d7-8545-08f2671a0e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.656916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c394d05-51bd-46d7-8545-08f2671a0e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:26.658565 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c394d05-51bd-46d7-8545-08f2671a0e3b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:26.673113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1334/2698] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:28:26.674338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61964126-dc84-44f3-9fb0-a7eae05c525c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:26.686215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61964126-dc84-44f3-9fb0-a7eae05c525c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:26.691019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eaddd-422f-42f8-bd7f-59fe0e443c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:26.691932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eaddd-422f-42f8-bd7f-59fe0e443c93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.692217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eaddd-422f-42f8-bd7f-59fe0e443c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.692497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eaddd-422f-42f8-bd7f-59fe0e443c93 admin admin] PATH_INFO: `/v3/projects/a018b1ddf0224770b73fbd574a34fbd8/users/a888807cfa6f4c9ca30e1021ed70dd4f/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.709438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891eaddd-422f-42f8-bd7f-59fe0e443c93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a018b1ddf0224770b73fbd574a34fbd8, user_id=a888807cfa6f4c9ca30e1021ed70dd4f, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:26.712445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891eaddd-422f-42f8-bd7f-59fe0e443c93 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:26.722248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:26.729467 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1365/2699] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:26 2022] PUT /identity/v3/projects/a018b1ddf0224770b73fbd574a34fbd8/users/a888807cfa6f4c9ca30e1021ed70dd4f/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:26.740755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d4e11-267a-4619-bf66-ce713a118b18 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.741251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d4e11-267a-4619-bf66-ce713a118b18 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.741566 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d4e11-267a-4619-bf66-ce713a118b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.776226 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-708d4e11-267a-4619-bf66-ce713a118b18 None None] MFA Rules not processed for user `a888807cfa6f4c9ca30e1021ed70dd4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:26.779509 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-708d4e11-267a-4619-bf66-ce713a118b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:26.859721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1335/2700] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:26.864332 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eaddd-422f-42f8-bd7f-59fe0e443c93 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:26.933497 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eaddd-422f-42f8-bd7f-59fe0e443c93 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:26.938167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c1d5b-fc3b-48c3-8630-6b0db6f6489e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:26.938851 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c1d5b-fc3b-48c3-8630-6b0db6f6489e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:26.939185 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c1d5b-fc3b-48c3-8630-6b0db6f6489e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:26.939468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c1d5b-fc3b-48c3-8630-6b0db6f6489e admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:26.940152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c1d5b-fc3b-48c3-8630-6b0db6f6489e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:26.942265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c1d5b-fc3b-48c3-8630-6b0db6f6489e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:26.970781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1366/2701] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:26 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:28:26.974792 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d4e11-267a-4619-bf66-ce713a118b18 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:26.985833 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d4e11-267a-4619-bf66-ce713a118b18 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:27.053512 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206e923-bef5-47ea-b7c6-c2cb9108f9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:27.054221 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206e923-bef5-47ea-b7c6-c2cb9108f9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:27.054503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206e923-bef5-47ea-b7c6-c2cb9108f9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:27.054774 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206e923-bef5-47ea-b7c6-c2cb9108f9a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:27.055767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206e923-bef5-47ea-b7c6-c2cb9108f9a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:27.058785 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206e923-bef5-47ea-b7c6-c2cb9108f9a8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:27.083759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1336/2702] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:26 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:28:27.086163 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c1d5b-fc3b-48c3-8630-6b0db6f6489e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:27.097790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c1d5b-fc3b-48c3-8630-6b0db6f6489e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:27.103928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086936d-08aa-416e-a558-1a538a098e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:27.104680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086936d-08aa-416e-a558-1a538a098e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:27.105031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086936d-08aa-416e-a558-1a538a098e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:27.105320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086936d-08aa-416e-a558-1a538a098e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:27.106381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086936d-08aa-416e-a558-1a538a098e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:27.107932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086936d-08aa-416e-a558-1a538a098e8b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:27.132522 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1367/2703] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:28:27.134016 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206e923-bef5-47ea-b7c6-c2cb9108f9a8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:27.144030 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206e923-bef5-47ea-b7c6-c2cb9108f9a8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:27.148714 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b205a9-8496-4e37-9838-dafb49622a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:27.149840 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b205a9-8496-4e37-9838-dafb49622a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:27.150193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b205a9-8496-4e37-9838-dafb49622a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:27.150474 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b205a9-8496-4e37-9838-dafb49622a36 admin admin] PATH_INFO: `/v3/projects/562b58a5d5eb4ea6843d414535ce6716/users/75ebefed7875425fbaf512bb66e183de/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:27.167431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b205a9-8496-4e37-9838-dafb49622a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=562b58a5d5eb4ea6843d414535ce6716, user_id=75ebefed7875425fbaf512bb66e183de, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:27.170170 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b205a9-8496-4e37-9838-dafb49622a36 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:27.184765 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1337/2704] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:27 2022] PUT /identity/v3/projects/562b58a5d5eb4ea6843d414535ce6716/users/75ebefed7875425fbaf512bb66e183de/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:27.191711 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086936d-08aa-416e-a558-1a538a098e8b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:27.202547 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086936d-08aa-416e-a558-1a538a098e8b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:27.272109 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631e9ce-7a56-4f2d-88e0-93a71b37a43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:27.272811 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631e9ce-7a56-4f2d-88e0-93a71b37a43e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:27.273043 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631e9ce-7a56-4f2d-88e0-93a71b37a43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:27.273335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631e9ce-7a56-4f2d-88e0-93a71b37a43e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:27.274084 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631e9ce-7a56-4f2d-88e0-93a71b37a43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:27.276316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631e9ce-7a56-4f2d-88e0-93a71b37a43e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:27.288127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1368/2705] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:27.295968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b205a9-8496-4e37-9838-dafb49622a36 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:27.302636 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:27.309296 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b205a9-8496-4e37-9838-dafb49622a36 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:27.323251 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a2c59-1fb6-45ea-9fc1-5d72730a1d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:27.324699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a2c59-1fb6-45ea-9fc1-5d72730a1d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:27.324699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a2c59-1fb6-45ea-9fc1-5d72730a1d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:27.324919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a2c59-1fb6-45ea-9fc1-5d72730a1d45 admin admin] PATH_INFO: `/v3/projects/562b58a5d5eb4ea6843d414535ce6716/users/75ebefed7875425fbaf512bb66e183de/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:27.330372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a2c59-1fb6-45ea-9fc1-5d72730a1d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=562b58a5d5eb4ea6843d414535ce6716, user_id=75ebefed7875425fbaf512bb66e183de, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:27.333080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a2c59-1fb6-45ea-9fc1-5d72730a1d45 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:27.359853 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1338/2706] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:27 2022] PUT /identity/v3/projects/562b58a5d5eb4ea6843d414535ce6716/users/75ebefed7875425fbaf512bb66e183de/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:27.359963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a2f59-ea60-46b0-aee2-69b230d40d67 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:27.359963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a2f59-ea60-46b0-aee2-69b230d40d67 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:27.359963 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a2f59-ea60-46b0-aee2-69b230d40d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:27.401617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3f7a2f59-ea60-46b0-aee2-69b230d40d67 None None] MFA Rules not processed for user `75ebefed7875425fbaf512bb66e183de`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:27.402996 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3f7a2f59-ea60-46b0-aee2-69b230d40d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:27.498263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:27.510511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1369/2707] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:27.531031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a2c59-1fb6-45ea-9fc1-5d72730a1d45 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:27.531031 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6d6a2c59-1fb6-45ea-9fc1-5d72730a1d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:27.603278 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a2c59-1fb6-45ea-9fc1-5d72730a1d45 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:27.611268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016346e5-4a5c-45ec-a505-6f96e67395e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:27.612025 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016346e5-4a5c-45ec-a505-6f96e67395e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:27.612230 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016346e5-4a5c-45ec-a505-6f96e67395e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:27.612547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016346e5-4a5c-45ec-a505-6f96e67395e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:27.624227 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016346e5-4a5c-45ec-a505-6f96e67395e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:27.625668 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016346e5-4a5c-45ec-a505-6f96e67395e4 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:27.636861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1339/2708] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:27.682375 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a2f59-ea60-46b0-aee2-69b230d40d67 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:27.684248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3f7a2f59-ea60-46b0-aee2-69b230d40d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:27.696988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a2f59-ea60-46b0-aee2-69b230d40d67 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:27.700747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e89a9-e183-440a-90ad-71538d27324c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:27.701398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e89a9-e183-440a-90ad-71538d27324c service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:27.701686 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e89a9-e183-440a-90ad-71538d27324c service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:27.701995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e89a9-e183-440a-90ad-71538d27324c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:27.717053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016346e5-4a5c-45ec-a505-6f96e67395e4 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:27.718653 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-016346e5-4a5c-45ec-a505-6f96e67395e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:27.765319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e89a9-e183-440a-90ad-71538d27324c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:27.766825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e89a9-e183-440a-90ad-71538d27324c service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:27.776072 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1370/2709] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:27.795151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016346e5-4a5c-45ec-a505-6f96e67395e4 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:27.802845 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee46d33-5a80-4e6e-8f72-f570b2b492a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:27.803849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee46d33-5a80-4e6e-8f72-f570b2b492a9 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:27.804283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee46d33-5a80-4e6e-8f72-f570b2b492a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:27.804691 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee46d33-5a80-4e6e-8f72-f570b2b492a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:27.807964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7ee46d33-5a80-4e6e-8f72-f570b2b492a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:27.825404 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee46d33-5a80-4e6e-8f72-f570b2b492a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:27.827089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee46d33-5a80-4e6e-8f72-f570b2b492a9 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:27.839552 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1340/2710] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:28:33.865602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e89a9-e183-440a-90ad-71538d27324c service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:33.930300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e89a9-e183-440a-90ad-71538d27324c service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:33.935876 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:33.936802 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:33.937039 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:33.937363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] PATH_INFO: `/v3/users/70d7248179af4af6b76e96ee46c9a94c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:33.938826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d7248179af4af6b76e96ee46c9a94c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:33.940784 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:33.987796 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d7248179af4af6b76e96ee46c9a94c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:33.988454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d7248179af4af6b76e96ee46c9a94c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:33.989064 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d7248179af4af6b76e96ee46c9a94c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.001616 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d7248179af4af6b76e96ee46c9a94c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.008374 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d7248179af4af6b76e96ee46c9a94c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.021177 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1371/2711] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:33 2022] DELETE /identity/v3/users/70d7248179af4af6b76e96ee46c9a94c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:34.026139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0fcf0-8e93-4525-a3c5-2c4075d0ee1b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:34.026605 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0fcf0-8e93-4525-a3c5-2c4075d0ee1b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:34.026947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0fcf0-8e93-4525-a3c5-2c4075d0ee1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:34.045155 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-13a0fcf0-8e93-4525-a3c5-2c4075d0ee1b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:34.046371 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-13a0fcf0-8e93-4525-a3c5-2c4075d0ee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:34.106138 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1341/2712] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:34.116971 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:34.118664 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:34.133122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:34.169799 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7410217-9b7f-48c0-b3fa-74da92c11ac9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:34.170212 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7410217-9b7f-48c0-b3fa-74da92c11ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:34.170369 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7410217-9b7f-48c0-b3fa-74da92c11ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:34.189749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a7410217-9b7f-48c0-b3fa-74da92c11ac9 None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:34.191649 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a7410217-9b7f-48c0-b3fa-74da92c11ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:34.193694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b40c6-81ca-48eb-aed7-9cd31b5330f8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:34.197980 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e134e-aae2-4059-9636-f7dd77212aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:34.199629 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e134e-aae2-4059-9636-f7dd77212aec service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:34.199964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e134e-aae2-4059-9636-f7dd77212aec service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:34.200291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e134e-aae2-4059-9636-f7dd77212aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:34.210314 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e134e-aae2-4059-9636-f7dd77212aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:34.211705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:34.211855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e134e-aae2-4059-9636-f7dd77212aec service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:34.223387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1372/2713] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:34.256120 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1342/2714] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:28:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:34.310879 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e134e-aae2-4059-9636-f7dd77212aec service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:34.312488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-107e134e-aae2-4059-9636-f7dd77212aec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:34.402122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e134e-aae2-4059-9636-f7dd77212aec service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:34.407373 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a4d6f-6815-422a-b293-463665bfea6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:34.408100 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a4d6f-6815-422a-b293-463665bfea6e service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:34.408476 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a4d6f-6815-422a-b293-463665bfea6e service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:34.408826 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a4d6f-6815-422a-b293-463665bfea6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:34.419842 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a4d6f-6815-422a-b293-463665bfea6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:34.421815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a4d6f-6815-422a-b293-463665bfea6e service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:34.434198 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1373/2715] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:28:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:28:34.562076 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7410217-9b7f-48c0-b3fa-74da92c11ac9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:34.568326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a7410217-9b7f-48c0-b3fa-74da92c11ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:34.613734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7410217-9b7f-48c0-b3fa-74da92c11ac9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:34.634631 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:34.636226 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:34.636949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:34.637480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] PATH_INFO: `/v3/projects/0a829404dbe14d25b69f5a453fed1fb6` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:34.639695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a829404dbe14d25b69f5a453fed1fb6)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:34.642368 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:34.716724 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a829404dbe14d25b69f5a453fed1fb6', 'request_id': 'req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.725095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a829404dbe14d25b69f5a453fed1fb6', 'request_id': 'req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.739758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a829404dbe14d25b69f5a453fed1fb6', 'request_id': 'req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.762207 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] The token cache is being invalidate because project 0a829404dbe14d25b69f5a453fed1fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:34.762818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.763934 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.766091 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1343/2716] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:34 2022] DELETE /identity/v3/projects/0a829404dbe14d25b69f5a453fed1fb6 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:34.772457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a4d6f-6815-422a-b293-463665bfea6e service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:34.789386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a4d6f-6815-422a-b293-463665bfea6e service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:34.910301 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:34.911443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:34.911727 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:34.912016 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] PATH_INFO: `/v3/users/a888807cfa6f4c9ca30e1021ed70dd4f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:34.913199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a888807cfa6f4c9ca30e1021ed70dd4f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:34.915178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:34.968899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:34.970435 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a888807cfa6f4c9ca30e1021ed70dd4f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.971304 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a888807cfa6f4c9ca30e1021ed70dd4f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.972101 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a888807cfa6f4c9ca30e1021ed70dd4f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.986830 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a888807cfa6f4c9ca30e1021ed70dd4f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:34.996360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a888807cfa6f4c9ca30e1021ed70dd4f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:35.018516 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1374/2717] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:34 2022] DELETE /identity/v3/users/a888807cfa6f4c9ca30e1021ed70dd4f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:35.312619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:35.315239 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:35.358917 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55b46d-0473-4b16-bdc0-47e39c0ed0a4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:35.476923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:35.479114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:35.480994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:35.481835 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] PATH_INFO: `/v3/projects/a018b1ddf0224770b73fbd574a34fbd8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:35.484547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a018b1ddf0224770b73fbd574a34fbd8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:35.486932 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:35.565107 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a018b1ddf0224770b73fbd574a34fbd8', 'request_id': 'req-1111fd46-6b9a-4df4-839e-52c01898d5a6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:35.567462 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:35.574739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a018b1ddf0224770b73fbd574a34fbd8', 'request_id': 'req-1111fd46-6b9a-4df4-839e-52c01898d5a6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:35.582947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a018b1ddf0224770b73fbd574a34fbd8', 'request_id': 'req-1111fd46-6b9a-4df4-839e-52c01898d5a6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:35.605562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] The token cache is being invalidate because project a018b1ddf0224770b73fbd574a34fbd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:35.606026 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:35.606806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:35.609776 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1344/2718] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:35 2022] DELETE /identity/v3/projects/a018b1ddf0224770b73fbd574a34fbd8 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:35.612788 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:35.615074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:35.634534 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8179076-4a46-4040-9654-5daaf9abe250 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:35.707864 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d4722-037e-4278-b22a-81d8deaefa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:35.709888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d4722-037e-4278-b22a-81d8deaefa98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:35.710395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d4722-037e-4278-b22a-81d8deaefa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:35.710737 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d4722-037e-4278-b22a-81d8deaefa98 admin admin] PATH_INFO: `/v3/users/75ebefed7875425fbaf512bb66e183de` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:35.712056 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d4722-037e-4278-b22a-81d8deaefa98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ebefed7875425fbaf512bb66e183de)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:35.714471 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d4722-037e-4278-b22a-81d8deaefa98 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:35.779417 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3d2d4722-037e-4278-b22a-81d8deaefa98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ebefed7875425fbaf512bb66e183de'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:35.780096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3d2d4722-037e-4278-b22a-81d8deaefa98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ebefed7875425fbaf512bb66e183de'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:35.780769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3d2d4722-037e-4278-b22a-81d8deaefa98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ebefed7875425fbaf512bb66e183de'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:35.795613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3d2d4722-037e-4278-b22a-81d8deaefa98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ebefed7875425fbaf512bb66e183de'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:35.803312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3d2d4722-037e-4278-b22a-81d8deaefa98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ebefed7875425fbaf512bb66e183de'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:35.819367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1375/2719] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:35 2022] DELETE /identity/v3/users/75ebefed7875425fbaf512bb66e183de => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:36.152482 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:36.155300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:36.175780 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111fd46-6b9a-4df4-839e-52c01898d5a6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:36.240806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:36.241751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:36.242040 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:36.242318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] PATH_INFO: `/v3/projects/562b58a5d5eb4ea6843d414535ce6716` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:36.243569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] RBAC: Authorizing `identity:delete_project(project_id=562b58a5d5eb4ea6843d414535ce6716)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:36.245538 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:36.314250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '562b58a5d5eb4ea6843d414535ce6716', 'request_id': 'req-8244441d-e626-4084-985d-107dfe9ac80f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:36.322926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '562b58a5d5eb4ea6843d414535ce6716', 'request_id': 'req-8244441d-e626-4084-985d-107dfe9ac80f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:36.330621 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '562b58a5d5eb4ea6843d414535ce6716', 'request_id': 'req-8244441d-e626-4084-985d-107dfe9ac80f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:36.349940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] The token cache is being invalidate because project 562b58a5d5eb4ea6843d414535ce6716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:36.350670 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:36.351648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8244441d-e626-4084-985d-107dfe9ac80f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:36.353412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1345/2720] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:36 2022] DELETE /identity/v3/projects/562b58a5d5eb4ea6843d414535ce6716 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:36.362970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a63abe-e43b-454e-913b-60673a8c0878 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:36.363623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a63abe-e43b-454e-913b-60673a8c0878 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:36.363623 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a63abe-e43b-454e-913b-60673a8c0878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:36.383988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-13a63abe-e43b-454e-913b-60673a8c0878 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:36.385323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-13a63abe-e43b-454e-913b-60673a8c0878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:36.430631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:36.462074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1376/2721] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:36.471960 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a516ed0-1957-4c35-99f2-40703ffb6cd4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:36.472280 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a516ed0-1957-4c35-99f2-40703ffb6cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:36.472526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a516ed0-1957-4c35-99f2-40703ffb6cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:36.492541 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4a516ed0-1957-4c35-99f2-40703ffb6cd4 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:36.493830 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4a516ed0-1957-4c35-99f2-40703ffb6cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:36.503793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1346/2722] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:36.508308 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a63abe-e43b-454e-913b-60673a8c0878 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:36.531492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a63abe-e43b-454e-913b-60673a8c0878 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:36.543706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff55c1-69b2-4b54-b420-710d56de2ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:36.544608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff55c1-69b2-4b54-b420-710d56de2ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:36.545024 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff55c1-69b2-4b54-b420-710d56de2ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:36.545326 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff55c1-69b2-4b54-b420-710d56de2ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:36.546472 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff55c1-69b2-4b54-b420-710d56de2ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:28:36.546952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff55c1-69b2-4b54-b420-710d56de2ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:36.549491 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff55c1-69b2-4b54-b420-710d56de2ba5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:36.568294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1377/2723] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:28:36.572502 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a516ed0-1957-4c35-99f2-40703ffb6cd4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:36.583758 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a516ed0-1957-4c35-99f2-40703ffb6cd4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:36.590005 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbceccc-7989-46e0-8dea-b78d36fc9e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:36.591377 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbceccc-7989-46e0-8dea-b78d36fc9e68 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:36.591748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbceccc-7989-46e0-8dea-b78d36fc9e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:36.592108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbceccc-7989-46e0-8dea-b78d36fc9e68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:36.592942 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbceccc-7989-46e0-8dea-b78d36fc9e68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:36.595204 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbceccc-7989-46e0-8dea-b78d36fc9e68 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:36.634660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1347/2724] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:36 2022] POST /identity/v3/projects => generated 359 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:36.638782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff55c1-69b2-4b54-b420-710d56de2ba5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:36.658955 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff55c1-69b2-4b54-b420-710d56de2ba5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:36.723349 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc9957-e8ba-49e3-a9f4-6f8a1e2d7989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:36.724051 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc9957-e8ba-49e3-a9f4-6f8a1e2d7989 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:36.724337 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc9957-e8ba-49e3-a9f4-6f8a1e2d7989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:36.724622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc9957-e8ba-49e3-a9f4-6f8a1e2d7989 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:36.725351 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc9957-e8ba-49e3-a9f4-6f8a1e2d7989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:36.727459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc9957-e8ba-49e3-a9f4-6f8a1e2d7989 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:36.752697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1378/2725] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:36 2022] POST /identity/v3/users => generated 280 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:36.754011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbceccc-7989-46e0-8dea-b78d36fc9e68 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:36.766293 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbceccc-7989-46e0-8dea-b78d36fc9e68 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:36.774412 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d4343-8745-4250-8b8d-9034e9668f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:36.776142 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d4343-8745-4250-8b8d-9034e9668f70 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:36.777700 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d4343-8745-4250-8b8d-9034e9668f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:36.778473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d4343-8745-4250-8b8d-9034e9668f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:36.780004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d4343-8745-4250-8b8d-9034e9668f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:36.783255 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d4343-8745-4250-8b8d-9034e9668f70 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:36.799546 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1348/2726] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:28:36.800769 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc9957-e8ba-49e3-a9f4-6f8a1e2d7989 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:36.816762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc9957-e8ba-49e3-a9f4-6f8a1e2d7989 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:36.822574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075d2d9-c0fd-4c3d-8f9e-2e61d72c2c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:36.823527 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075d2d9-c0fd-4c3d-8f9e-2e61d72c2c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:36.824222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075d2d9-c0fd-4c3d-8f9e-2e61d72c2c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:36.824633 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075d2d9-c0fd-4c3d-8f9e-2e61d72c2c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:36.825649 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075d2d9-c0fd-4c3d-8f9e-2e61d72c2c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:36.827511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075d2d9-c0fd-4c3d-8f9e-2e61d72c2c94 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:36.841679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1379/2727] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:36.846525 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d4343-8745-4250-8b8d-9034e9668f70 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:36.859267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d4343-8745-4250-8b8d-9034e9668f70 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:36.864814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a2ca2-8ff4-46e5-9e57-d98354e4d9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:36.864814 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a2ca2-8ff4-46e5-9e57-d98354e4d9b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:36.865059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a2ca2-8ff4-46e5-9e57-d98354e4d9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:36.865698 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a2ca2-8ff4-46e5-9e57-d98354e4d9b0 admin admin] PATH_INFO: `/v3/projects/91e5797e738e4471bec3cae160fef804/users/b947dc69a25443319938c1cc9a5898e1/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:36.890352 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a2ca2-8ff4-46e5-9e57-d98354e4d9b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e5797e738e4471bec3cae160fef804, user_id=b947dc69a25443319938c1cc9a5898e1, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:36.893963 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a2ca2-8ff4-46e5-9e57-d98354e4d9b0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:36.911281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1349/2728] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:36 2022] PUT /identity/v3/projects/91e5797e738e4471bec3cae160fef804/users/b947dc69a25443319938c1cc9a5898e1/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:36.923160 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7987be-3afd-481a-8e82-bb8a0be6a5a8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:36.923807 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7987be-3afd-481a-8e82-bb8a0be6a5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:36.924121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7987be-3afd-481a-8e82-bb8a0be6a5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:36.959812 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cb7987be-3afd-481a-8e82-bb8a0be6a5a8 None None] MFA Rules not processed for user `b947dc69a25443319938c1cc9a5898e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:36.961019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cb7987be-3afd-481a-8e82-bb8a0be6a5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:37.034999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1380/2729] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:37.067971 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a2ca2-8ff4-46e5-9e57-d98354e4d9b0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:37.070514 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-774a2ca2-8ff4-46e5-9e57-d98354e4d9b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:37.081188 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:37.143872 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a2ca2-8ff4-46e5-9e57-d98354e4d9b0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:37.150935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ce709-aeca-449e-9569-156e087f4374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:37.151903 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ce709-aeca-449e-9569-156e087f4374 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:37.152302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ce709-aeca-449e-9569-156e087f4374 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:37.152519 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ce709-aeca-449e-9569-156e087f4374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:37.163947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ce709-aeca-449e-9569-156e087f4374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:37.165537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ce709-aeca-449e-9569-156e087f4374 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:37.176516 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1350/2730] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:37 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:37.196305 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7987be-3afd-481a-8e82-bb8a0be6a5a8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:37.251277 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:37.293622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7987be-3afd-481a-8e82-bb8a0be6a5a8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:37.299162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:37.300124 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:37.300411 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:37.300694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] PATH_INFO: `/v3/users/b947dc69a25443319938c1cc9a5898e1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:37.302145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b947dc69a25443319938c1cc9a5898e1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:37.304204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:37.367857 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b947dc69a25443319938c1cc9a5898e1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:37.368576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b947dc69a25443319938c1cc9a5898e1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:37.369270 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b947dc69a25443319938c1cc9a5898e1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:37.384959 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b947dc69a25443319938c1cc9a5898e1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:37.393012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b947dc69a25443319938c1cc9a5898e1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:37.408042 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1381/2731] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:37 2022] DELETE /identity/v3/users/b947dc69a25443319938c1cc9a5898e1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:37.412908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6fbe9-20d0-4338-8d8b-2b40bdc20534 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:37.413382 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6fbe9-20d0-4338-8d8b-2b40bdc20534 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:37.413466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6fbe9-20d0-4338-8d8b-2b40bdc20534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:37.457001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-66c6fbe9-20d0-4338-8d8b-2b40bdc20534 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:37.459472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-66c6fbe9-20d0-4338-8d8b-2b40bdc20534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:37.548307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1351/2732] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:37.559940 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:37.561455 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:37.642136 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c556e3d-ff36-47f4-a799-c94c8653ce34 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:37.646426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe656461-f131-412f-84a7-dfa8c7615327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:37.647322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe656461-f131-412f-84a7-dfa8c7615327 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:37.647798 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe656461-f131-412f-84a7-dfa8c7615327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:37.648273 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe656461-f131-412f-84a7-dfa8c7615327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:37.662181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe656461-f131-412f-84a7-dfa8c7615327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:37.663585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe656461-f131-412f-84a7-dfa8c7615327 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:37.674957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1382/2733] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:38.014037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6fbe9-20d0-4338-8d8b-2b40bdc20534 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:38.015642 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-66c6fbe9-20d0-4338-8d8b-2b40bdc20534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:38.040372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6fbe9-20d0-4338-8d8b-2b40bdc20534 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:38.050230 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:38.051521 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:38.051860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:38.052258 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] PATH_INFO: `/v3/projects/91e5797e738e4471bec3cae160fef804` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:38.053773 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e5797e738e4471bec3cae160fef804)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:38.056301 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:38.124051 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e5797e738e4471bec3cae160fef804', 'request_id': 'req-def8201e-9f9a-41f8-8ef0-e4aee3712923'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:38.133065 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e5797e738e4471bec3cae160fef804', 'request_id': 'req-def8201e-9f9a-41f8-8ef0-e4aee3712923'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:38.142120 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e5797e738e4471bec3cae160fef804', 'request_id': 'req-def8201e-9f9a-41f8-8ef0-e4aee3712923'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:38.156505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] The token cache is being invalidate because project 91e5797e738e4471bec3cae160fef804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:38.157143 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:38.158056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-def8201e-9f9a-41f8-8ef0-e4aee3712923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:38.160611 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1352/2734] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:38 2022] DELETE /identity/v3/projects/91e5797e738e4471bec3cae160fef804 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:38.165992 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bda175-3fd5-417d-b0c7-2323ca4c2767 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:38.166566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bda175-3fd5-417d-b0c7-2323ca4c2767 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:38.167350 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bda175-3fd5-417d-b0c7-2323ca4c2767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:38.190762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-58bda175-3fd5-417d-b0c7-2323ca4c2767 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:38.192306 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-58bda175-3fd5-417d-b0c7-2323ca4c2767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:38.257109 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:38.268557 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1383/2735] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:38.277321 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40bc9f-1d1b-446c-89a5-9f8d4ed772ef None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:38.277900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40bc9f-1d1b-446c-89a5-9f8d4ed772ef None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:38.278044 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40bc9f-1d1b-446c-89a5-9f8d4ed772ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:38.301178 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cf40bc9f-1d1b-446c-89a5-9f8d4ed772ef None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:38.302479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf40bc9f-1d1b-446c-89a5-9f8d4ed772ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:38.312598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1353/2736] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:38.316935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bda175-3fd5-417d-b0c7-2323ca4c2767 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:38.328642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bda175-3fd5-417d-b0c7-2323ca4c2767 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:38.334141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458e73b-8849-4baa-a9ec-fae876f8bda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:38.334913 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458e73b-8849-4baa-a9ec-fae876f8bda1 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:38.335239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458e73b-8849-4baa-a9ec-fae876f8bda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:38.335529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458e73b-8849-4baa-a9ec-fae876f8bda1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:38.336262 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458e73b-8849-4baa-a9ec-fae876f8bda1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:28:38.336572 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458e73b-8849-4baa-a9ec-fae876f8bda1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:38.338202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458e73b-8849-4baa-a9ec-fae876f8bda1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:38.356633 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1384/2737] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:28:38.361116 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40bc9f-1d1b-446c-89a5-9f8d4ed772ef None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:38.372912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40bc9f-1d1b-446c-89a5-9f8d4ed772ef None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:38.378413 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ef65e-a73a-49b2-976f-843a21f9ac5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:38.379098 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ef65e-a73a-49b2-976f-843a21f9ac5f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:38.379399 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ef65e-a73a-49b2-976f-843a21f9ac5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:38.379688 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ef65e-a73a-49b2-976f-843a21f9ac5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:38.380391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ef65e-a73a-49b2-976f-843a21f9ac5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:38.382421 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ef65e-a73a-49b2-976f-843a21f9ac5f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:38.416267 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1354/2738] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:38 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:38.420935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458e73b-8849-4baa-a9ec-fae876f8bda1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:38.432902 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458e73b-8849-4baa-a9ec-fae876f8bda1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:38.498320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6dd0e2-08de-4b25-ab33-298d7b9497d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:38.498988 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dd0e2-08de-4b25-ab33-298d7b9497d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:38.499278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dd0e2-08de-4b25-ab33-298d7b9497d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:38.499557 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dd0e2-08de-4b25-ab33-298d7b9497d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:38.500240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6dd0e2-08de-4b25-ab33-298d7b9497d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:38.502387 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6dd0e2-08de-4b25-ab33-298d7b9497d3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:38.530269 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1385/2739] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:38 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:38.531691 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ef65e-a73a-49b2-976f-843a21f9ac5f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:38.545954 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:38.551018 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ef65e-a73a-49b2-976f-843a21f9ac5f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:38.563103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f27ed-0c1f-45c1-869a-a51aa03ae338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:38.564920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f27ed-0c1f-45c1-869a-a51aa03ae338 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:38.567888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f27ed-0c1f-45c1-869a-a51aa03ae338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:38.567888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f27ed-0c1f-45c1-869a-a51aa03ae338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:38.569233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f27ed-0c1f-45c1-869a-a51aa03ae338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:38.571371 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f27ed-0c1f-45c1-869a-a51aa03ae338 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:38.589594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1355/2740] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:38.591795 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6dd0e2-08de-4b25-ab33-298d7b9497d3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:38.603379 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6dd0e2-08de-4b25-ab33-298d7b9497d3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:38.608514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c53857-c265-4c09-b30a-4d2be652b1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:38.609205 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c53857-c265-4c09-b30a-4d2be652b1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:38.609495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c53857-c265-4c09-b30a-4d2be652b1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:38.609823 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c53857-c265-4c09-b30a-4d2be652b1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:38.610929 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c53857-c265-4c09-b30a-4d2be652b1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:38.612508 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c53857-c265-4c09-b30a-4d2be652b1e9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:38.624450 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1386/2741] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:38.628569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f27ed-0c1f-45c1-869a-a51aa03ae338 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:38.639894 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f27ed-0c1f-45c1-869a-a51aa03ae338 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:38.644870 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b28e9-86de-4ba2-b5b7-e5220d2cf4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:38.645900 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b28e9-86de-4ba2-b5b7-e5220d2cf4e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:38.646403 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b28e9-86de-4ba2-b5b7-e5220d2cf4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:38.646753 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b28e9-86de-4ba2-b5b7-e5220d2cf4e7 admin admin] PATH_INFO: `/v3/projects/297d9ce68caa4821875a6bf6a775ed64/users/a5e8cbee18ee4ec2aac08cc18e19764e/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:38.662664 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b28e9-86de-4ba2-b5b7-e5220d2cf4e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=297d9ce68caa4821875a6bf6a775ed64, user_id=a5e8cbee18ee4ec2aac08cc18e19764e, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:38.665526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b28e9-86de-4ba2-b5b7-e5220d2cf4e7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:38.689992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1356/2742] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:38 2022] PUT /identity/v3/projects/297d9ce68caa4821875a6bf6a775ed64/users/a5e8cbee18ee4ec2aac08cc18e19764e/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:38.691739 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be849f0-5c94-44f9-b75d-98a14321cbf7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:38.692192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be849f0-5c94-44f9-b75d-98a14321cbf7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:38.692576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be849f0-5c94-44f9-b75d-98a14321cbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:38.721978 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5be849f0-5c94-44f9-b75d-98a14321cbf7 None None] MFA Rules not processed for user `a5e8cbee18ee4ec2aac08cc18e19764e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:38.723324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5be849f0-5c94-44f9-b75d-98a14321cbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:38.798093 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1387/2743] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:38.811210 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b28e9-86de-4ba2-b5b7-e5220d2cf4e7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:38.813154 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf9b28e9-86de-4ba2-b5b7-e5220d2cf4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:38.902256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b28e9-86de-4ba2-b5b7-e5220d2cf4e7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:38.910486 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856c189-ea48-4436-baca-08133056d2fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:38.910975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856c189-ea48-4436-baca-08133056d2fe service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:38.911530 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856c189-ea48-4436-baca-08133056d2fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:38.911858 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856c189-ea48-4436-baca-08133056d2fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:38.922431 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856c189-ea48-4436-baca-08133056d2fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:38.923977 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856c189-ea48-4436-baca-08133056d2fe service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:38.937089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1357/2744] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:42.217265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be849f0-5c94-44f9-b75d-98a14321cbf7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:42.315878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:42.322475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be849f0-5c94-44f9-b75d-98a14321cbf7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:42.331656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:42.332639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:42.333002 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:42.333376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] PATH_INFO: `/v3/users/a5e8cbee18ee4ec2aac08cc18e19764e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:42.334926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5e8cbee18ee4ec2aac08cc18e19764e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:42.337068 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:42.396587 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5e8cbee18ee4ec2aac08cc18e19764e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:42.397234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5e8cbee18ee4ec2aac08cc18e19764e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:42.397906 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5e8cbee18ee4ec2aac08cc18e19764e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:42.412263 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5e8cbee18ee4ec2aac08cc18e19764e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:42.421265 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5e8cbee18ee4ec2aac08cc18e19764e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:42.442995 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1388/2745] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:42 2022] DELETE /identity/v3/users/a5e8cbee18ee4ec2aac08cc18e19764e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:42.447349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18270e-0413-46e1-aaba-1ad9ed716e7b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:42.447844 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18270e-0413-46e1-aaba-1ad9ed716e7b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:42.447949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18270e-0413-46e1-aaba-1ad9ed716e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:42.470523 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6f18270e-0413-46e1-aaba-1ad9ed716e7b None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:42.472470 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6f18270e-0413-46e1-aaba-1ad9ed716e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:42.551417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1358/2746] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:42.560397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:42.561853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:42.627577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71827e-50a9-4d04-ab1b-d5c7c38b4a78 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:42.633204 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c52a14-de23-4ca8-b2cb-c30c0154eb04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:42.633887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c52a14-de23-4ca8-b2cb-c30c0154eb04 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:42.634170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c52a14-de23-4ca8-b2cb-c30c0154eb04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:42.634475 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c52a14-de23-4ca8-b2cb-c30c0154eb04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:42.645395 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c52a14-de23-4ca8-b2cb-c30c0154eb04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:42.646954 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c52a14-de23-4ca8-b2cb-c30c0154eb04 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:42.657533 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1389/2747] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:42.987816 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18270e-0413-46e1-aaba-1ad9ed716e7b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:42.989487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6f18270e-0413-46e1-aaba-1ad9ed716e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:43.015846 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18270e-0413-46e1-aaba-1ad9ed716e7b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:43.027981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:43.028892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:43.029686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:43.029686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] PATH_INFO: `/v3/projects/297d9ce68caa4821875a6bf6a775ed64` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:43.035127 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=297d9ce68caa4821875a6bf6a775ed64)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:43.038052 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:43.111860 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297d9ce68caa4821875a6bf6a775ed64', 'request_id': 'req-c4bf9bc7-494f-45b7-861d-d00890ae5999'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:43.117634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297d9ce68caa4821875a6bf6a775ed64', 'request_id': 'req-c4bf9bc7-494f-45b7-861d-d00890ae5999'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:43.125940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297d9ce68caa4821875a6bf6a775ed64', 'request_id': 'req-c4bf9bc7-494f-45b7-861d-d00890ae5999'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:43.130095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:43.152528 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] The token cache is being invalidate because project 297d9ce68caa4821875a6bf6a775ed64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:43.153110 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:43.153980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c4bf9bc7-494f-45b7-861d-d00890ae5999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:43.155913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1359/2748] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:42 2022] DELETE /identity/v3/projects/297d9ce68caa4821875a6bf6a775ed64 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:43.162088 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b347f-1956-4d78-b6c7-4a951544467f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:43.162386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b347f-1956-4d78-b6c7-4a951544467f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:43.162743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b347f-1956-4d78-b6c7-4a951544467f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:43.187634 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f26b347f-1956-4d78-b6c7-4a951544467f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:43.188953 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f26b347f-1956-4d78-b6c7-4a951544467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:43.256145 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1390/2749] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:43.264732 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5aa3a0-cb61-4754-b60a-f3a24af93838 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:43.265017 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5aa3a0-cb61-4754-b60a-f3a24af93838 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:43.265298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5aa3a0-cb61-4754-b60a-f3a24af93838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:43.284943 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-de5aa3a0-cb61-4754-b60a-f3a24af93838 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:43.286439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-de5aa3a0-cb61-4754-b60a-f3a24af93838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:43.294749 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1360/2750] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:43.298990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b347f-1956-4d78-b6c7-4a951544467f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:43.310141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b347f-1956-4d78-b6c7-4a951544467f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:43.315248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2000b-a3e6-4c08-950a-8f70e2c1cd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:43.315957 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2000b-a3e6-4c08-950a-8f70e2c1cd38 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:43.316208 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2000b-a3e6-4c08-950a-8f70e2c1cd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:43.316486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2000b-a3e6-4c08-950a-8f70e2c1cd38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:43.317256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2000b-a3e6-4c08-950a-8f70e2c1cd38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:28:43.317563 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2000b-a3e6-4c08-950a-8f70e2c1cd38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:43.319211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2000b-a3e6-4c08-950a-8f70e2c1cd38 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:43.336561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1391/2751] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:28:43.338124 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5aa3a0-cb61-4754-b60a-f3a24af93838 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:43.348721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5aa3a0-cb61-4754-b60a-f3a24af93838 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:43.355111 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4889d7-c826-4e45-948c-ecc1d7b18186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:43.355863 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4889d7-c826-4e45-948c-ecc1d7b18186 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:43.356145 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4889d7-c826-4e45-948c-ecc1d7b18186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:43.356422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4889d7-c826-4e45-948c-ecc1d7b18186 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:43.357106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4889d7-c826-4e45-948c-ecc1d7b18186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:43.359054 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4889d7-c826-4e45-948c-ecc1d7b18186 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:43.396897 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1361/2752] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:43 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:43.400947 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2000b-a3e6-4c08-950a-8f70e2c1cd38 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:43.409057 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:43.414006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2000b-a3e6-4c08-950a-8f70e2c1cd38 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:43.503968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb31bf7-4560-4141-9d30-65f389d2002a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:43.504746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb31bf7-4560-4141-9d30-65f389d2002a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:43.505074 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb31bf7-4560-4141-9d30-65f389d2002a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:43.505343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb31bf7-4560-4141-9d30-65f389d2002a admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:43.506156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb31bf7-4560-4141-9d30-65f389d2002a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:43.508227 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb31bf7-4560-4141-9d30-65f389d2002a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:43.541917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1392/2753] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:43 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:43.546580 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4889d7-c826-4e45-948c-ecc1d7b18186 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:43.558473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4889d7-c826-4e45-948c-ecc1d7b18186 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:43.565060 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08302140-6a18-4d1e-8586-3e205a7d1d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:43.565882 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08302140-6a18-4d1e-8586-3e205a7d1d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:43.566407 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08302140-6a18-4d1e-8586-3e205a7d1d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:43.567033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08302140-6a18-4d1e-8586-3e205a7d1d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:43.568433 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08302140-6a18-4d1e-8586-3e205a7d1d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:43.570755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08302140-6a18-4d1e-8586-3e205a7d1d09 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:43.585438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1362/2754] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:43.590025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb31bf7-4560-4141-9d30-65f389d2002a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:43.602223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb31bf7-4560-4141-9d30-65f389d2002a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:43.609117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3097130-fdad-41b3-a86d-cc9a032d2318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:43.609117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3097130-fdad-41b3-a86d-cc9a032d2318 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:43.609117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3097130-fdad-41b3-a86d-cc9a032d2318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:43.609117 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3097130-fdad-41b3-a86d-cc9a032d2318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:43.609743 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3097130-fdad-41b3-a86d-cc9a032d2318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:43.611323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3097130-fdad-41b3-a86d-cc9a032d2318 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:43.625628 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1393/2755] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:43.629678 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08302140-6a18-4d1e-8586-3e205a7d1d09 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:43.650739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08302140-6a18-4d1e-8586-3e205a7d1d09 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:43.658159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90836fe-a3da-4903-9766-4260143a884f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:43.659181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90836fe-a3da-4903-9766-4260143a884f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:43.659544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90836fe-a3da-4903-9766-4260143a884f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:43.659893 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90836fe-a3da-4903-9766-4260143a884f admin admin] PATH_INFO: `/v3/projects/c8efc0621a1242329970cda13b3fee4b/users/fcca87cd01ae464cb1484c3bbd52aaf6/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:43.685259 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90836fe-a3da-4903-9766-4260143a884f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8efc0621a1242329970cda13b3fee4b, user_id=fcca87cd01ae464cb1484c3bbd52aaf6, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:43.689584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90836fe-a3da-4903-9766-4260143a884f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:43.711135 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1363/2756] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:43 2022] PUT /identity/v3/projects/c8efc0621a1242329970cda13b3fee4b/users/fcca87cd01ae464cb1484c3bbd52aaf6/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:43.713127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e00d49-6fb2-480a-ba94-d964da7a172f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:43.713494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e00d49-6fb2-480a-ba94-d964da7a172f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:43.714433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e00d49-6fb2-480a-ba94-d964da7a172f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:43.735940 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f8e00d49-6fb2-480a-ba94-d964da7a172f None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:43.737304 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f8e00d49-6fb2-480a-ba94-d964da7a172f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:43.831192 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1394/2757] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:43.845222 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90836fe-a3da-4903-9766-4260143a884f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:43.847394 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f90836fe-a3da-4903-9766-4260143a884f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:43.938192 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90836fe-a3da-4903-9766-4260143a884f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:43.942001 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10667283-89cb-4163-8abd-2d56ed2933c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:43.943358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10667283-89cb-4163-8abd-2d56ed2933c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:43.943718 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10667283-89cb-4163-8abd-2d56ed2933c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:43.944079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10667283-89cb-4163-8abd-2d56ed2933c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:43.957741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10667283-89cb-4163-8abd-2d56ed2933c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:43.959307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10667283-89cb-4163-8abd-2d56ed2933c0 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:43.973008 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1364/2758] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:49.006444 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a3657-dfdb-4c03-bd40-3895ceaf4596 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:49.007023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a3657-dfdb-4c03-bd40-3895ceaf4596 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:49.007023 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a3657-dfdb-4c03-bd40-3895ceaf4596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:49.045697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c89a3657-dfdb-4c03-bd40-3895ceaf4596 None None] MFA Rules not processed for user `fcca87cd01ae464cb1484c3bbd52aaf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:49.047073 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c89a3657-dfdb-4c03-bd40-3895ceaf4596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:49.145060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1395/2759] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:49.149851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10667283-89cb-4163-8abd-2d56ed2933c0 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:49.165667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10667283-89cb-4163-8abd-2d56ed2933c0 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:49.183284 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271544c3-3605-4313-9869-06069497eb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:49.184562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271544c3-3605-4313-9869-06069497eb53 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:49.185172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271544c3-3605-4313-9869-06069497eb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:49.185689 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271544c3-3605-4313-9869-06069497eb53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:49.186576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271544c3-3605-4313-9869-06069497eb53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:49.188666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271544c3-3605-4313-9869-06069497eb53 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:49.252739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1365/2760] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:49 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:49.257847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a3657-dfdb-4c03-bd40-3895ceaf4596 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:49.280367 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a3657-dfdb-4c03-bd40-3895ceaf4596 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:49.294440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:49.406926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c20bd0-7536-47d5-929e-8ff80dc8eb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:49.407633 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c20bd0-7536-47d5-929e-8ff80dc8eb17 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:49.408061 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c20bd0-7536-47d5-929e-8ff80dc8eb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:49.408349 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c20bd0-7536-47d5-929e-8ff80dc8eb17 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:49.409152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c20bd0-7536-47d5-929e-8ff80dc8eb17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:49.411197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c20bd0-7536-47d5-929e-8ff80dc8eb17 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:49.467206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1396/2761] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:49 2022] POST /identity/v3/users => generated 299 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:49.468789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271544c3-3605-4313-9869-06069497eb53 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:49.486272 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271544c3-3605-4313-9869-06069497eb53 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:49.503771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718b224-d894-4111-a141-fd60b1493555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:49.504472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718b224-d894-4111-a141-fd60b1493555 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:49.504772 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718b224-d894-4111-a141-fd60b1493555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:49.505086 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718b224-d894-4111-a141-fd60b1493555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:49.505937 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718b224-d894-4111-a141-fd60b1493555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:49.507876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718b224-d894-4111-a141-fd60b1493555 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:49.545822 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1366/2762] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:49.550019 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c20bd0-7536-47d5-929e-8ff80dc8eb17 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:49.576591 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c20bd0-7536-47d5-929e-8ff80dc8eb17 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:49.584368 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3338b5d-e840-4db7-aa16-9c8e01af07b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:49.585602 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338b5d-e840-4db7-aa16-9c8e01af07b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:49.586065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338b5d-e840-4db7-aa16-9c8e01af07b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:49.586510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338b5d-e840-4db7-aa16-9c8e01af07b4 admin admin] PATH_INFO: `/v3/projects/ac6390639b1e42bdbdb83766fea23486/users/a8fb914d98984987b928e79d4b75aa4d/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:49.624631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3338b5d-e840-4db7-aa16-9c8e01af07b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6390639b1e42bdbdb83766fea23486, user_id=a8fb914d98984987b928e79d4b75aa4d, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:49.627710 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3338b5d-e840-4db7-aa16-9c8e01af07b4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:49.655666 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1397/2763] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:49 2022] PUT /identity/v3/projects/ac6390639b1e42bdbdb83766fea23486/users/a8fb914d98984987b928e79d4b75aa4d/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:49.662312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718b224-d894-4111-a141-fd60b1493555 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:49.681680 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718b224-d894-4111-a141-fd60b1493555 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:49.784803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645616cf-572a-4683-8add-3f439b5a27af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:49.785494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645616cf-572a-4683-8add-3f439b5a27af admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:49.785766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645616cf-572a-4683-8add-3f439b5a27af admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:49.786174 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645616cf-572a-4683-8add-3f439b5a27af admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:49.787026 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645616cf-572a-4683-8add-3f439b5a27af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:49.788631 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645616cf-572a-4683-8add-3f439b5a27af admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:49.794080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:49.810740 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3338b5d-e840-4db7-aa16-9c8e01af07b4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:49.811395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1367/2764] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:49.824440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3338b5d-e840-4db7-aa16-9c8e01af07b4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:49.829867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946faf36-9e9b-4e50-ad97-d02639254537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:49.830775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946faf36-9e9b-4e50-ad97-d02639254537 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:49.831055 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946faf36-9e9b-4e50-ad97-d02639254537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:49.831328 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946faf36-9e9b-4e50-ad97-d02639254537 admin admin] PATH_INFO: `/v3/projects/ac6390639b1e42bdbdb83766fea23486/users/a8fb914d98984987b928e79d4b75aa4d/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:49.833473 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946faf36-9e9b-4e50-ad97-d02639254537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6390639b1e42bdbdb83766fea23486, user_id=a8fb914d98984987b928e79d4b75aa4d, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:49.836391 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946faf36-9e9b-4e50-ad97-d02639254537 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:49.854683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1398/2765] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:49 2022] PUT /identity/v3/projects/ac6390639b1e42bdbdb83766fea23486/users/a8fb914d98984987b928e79d4b75aa4d/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:52.195160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c2684-be94-4fea-b142-101980c0cfb5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:52.196187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c2684-be94-4fea-b142-101980c0cfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:52.196734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c2684-be94-4fea-b142-101980c0cfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:52.223487 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-361c2684-be94-4fea-b142-101980c0cfb5 None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:52.224972 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-361c2684-be94-4fea-b142-101980c0cfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:52.323918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1368/2766] 10.222.0.42 () {58 vars in 967 bytes} [Wed May 25 16:28:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:52.331968 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946faf36-9e9b-4e50-ad97-d02639254537 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:52.334406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-946faf36-9e9b-4e50-ad97-d02639254537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:52.420938 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c2684-be94-4fea-b142-101980c0cfb5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:52.423663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-361c2684-be94-4fea-b142-101980c0cfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:52.454831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946faf36-9e9b-4e50-ad97-d02639254537 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:52.462594 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01f792-9da1-45a6-9cac-21b835b69329 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:52.463492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01f792-9da1-45a6-9cac-21b835b69329 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:52.463884 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01f792-9da1-45a6-9cac-21b835b69329 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:52.464246 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01f792-9da1-45a6-9cac-21b835b69329 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:52.474347 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01f792-9da1-45a6-9cac-21b835b69329 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:52.476243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01f792-9da1-45a6-9cac-21b835b69329 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:52.488647 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1399/2767] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:28:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:28:52.581512 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c2684-be94-4fea-b142-101980c0cfb5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:52.590507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:52.591430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:52.593936 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:52.594771 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] PATH_INFO: `/v3/users/2aeadda12d6546449852095fb85e072b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:52.597504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aeadda12d6546449852095fb85e072b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:52.601395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:52.747324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aeadda12d6546449852095fb85e072b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:52.766906 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aeadda12d6546449852095fb85e072b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:52.768093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aeadda12d6546449852095fb85e072b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:52.805491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aeadda12d6546449852095fb85e072b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:52.823876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aeadda12d6546449852095fb85e072b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:52.836731 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1369/2768] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:52 2022] DELETE /identity/v3/users/2aeadda12d6546449852095fb85e072b => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:52.839854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7be806-590d-448e-94b1-0c25311cb821 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:52.840238 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7be806-590d-448e-94b1-0c25311cb821 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:52.840592 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7be806-590d-448e-94b1-0c25311cb821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:52.886646 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0c7be806-590d-448e-94b1-0c25311cb821 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:52.887818 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0c7be806-590d-448e-94b1-0c25311cb821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:52.981610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:53.002353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1400/2769] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:53.010325 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:53.013147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:53.105229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:53.175867 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74c7dc-6b2e-4135-a312-f60e09ca1c16 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:53.182329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180df5e-2fca-43e5-bd9f-3314ea4ae8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:53.183307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180df5e-2fca-43e5-bd9f-3314ea4ae8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:53.183508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180df5e-2fca-43e5-bd9f-3314ea4ae8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:53.183752 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180df5e-2fca-43e5-bd9f-3314ea4ae8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:53.195919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180df5e-2fca-43e5-bd9f-3314ea4ae8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:53.199191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180df5e-2fca-43e5-bd9f-3314ea4ae8a8 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:53.211220 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1370/2770] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:53.962478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7be806-590d-448e-94b1-0c25311cb821 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:53.965066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0c7be806-590d-448e-94b1-0c25311cb821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:53.984982 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7be806-590d-448e-94b1-0c25311cb821 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:53.996542 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:53.999323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:53.999701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:54.000071 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] PATH_INFO: `/v3/projects/f65875f3e7b640b799d1ddf5c88e212d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:54.002512 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f65875f3e7b640b799d1ddf5c88e212d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:54.006196 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:54.075266 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f65875f3e7b640b799d1ddf5c88e212d', 'request_id': 'req-68a8d809-3902-480e-ab46-888f390051f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:54.109003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f65875f3e7b640b799d1ddf5c88e212d', 'request_id': 'req-68a8d809-3902-480e-ab46-888f390051f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:54.121606 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f65875f3e7b640b799d1ddf5c88e212d', 'request_id': 'req-68a8d809-3902-480e-ab46-888f390051f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:54.137012 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] The token cache is being invalidate because project f65875f3e7b640b799d1ddf5c88e212d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:54.137636 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:54.139156 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68a8d809-3902-480e-ab46-888f390051f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:54.146449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1401/2771] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:53 2022] DELETE /identity/v3/projects/f65875f3e7b640b799d1ddf5c88e212d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:54.147887 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efb63-eac5-4e3e-a3dc-d0ce617cbb61 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:54.148033 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efb63-eac5-4e3e-a3dc-d0ce617cbb61 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:54.148589 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efb63-eac5-4e3e-a3dc-d0ce617cbb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:54.178285 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-653efb63-eac5-4e3e-a3dc-d0ce617cbb61 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:54.179927 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-653efb63-eac5-4e3e-a3dc-d0ce617cbb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:54.277807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1371/2772] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:54.287700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b617932-3421-402a-a635-06bae748f9ab None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:54.288006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b617932-3421-402a-a635-06bae748f9ab None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:54.288412 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b617932-3421-402a-a635-06bae748f9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:54.323786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4b617932-3421-402a-a635-06bae748f9ab None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:54.327780 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4b617932-3421-402a-a635-06bae748f9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:54.338009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1402/2773] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:54.342866 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efb63-eac5-4e3e-a3dc-d0ce617cbb61 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:54.357052 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efb63-eac5-4e3e-a3dc-d0ce617cbb61 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:54.367778 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d49c4-2277-429d-8ac1-3da0b94147cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:54.368609 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d49c4-2277-429d-8ac1-3da0b94147cc admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:54.370256 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d49c4-2277-429d-8ac1-3da0b94147cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:54.370793 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d49c4-2277-429d-8ac1-3da0b94147cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:54.372859 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d49c4-2277-429d-8ac1-3da0b94147cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:28:54.374442 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d49c4-2277-429d-8ac1-3da0b94147cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:54.377576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d49c4-2277-429d-8ac1-3da0b94147cc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:54.401597 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1372/2774] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:28:54.401853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b617932-3421-402a-a635-06bae748f9ab None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:54.414859 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b617932-3421-402a-a635-06bae748f9ab None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:54.421585 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec60d2a-1a1e-4213-81b6-3a9a62a02f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:54.422388 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec60d2a-1a1e-4213-81b6-3a9a62a02f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:54.422928 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec60d2a-1a1e-4213-81b6-3a9a62a02f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:54.423213 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec60d2a-1a1e-4213-81b6-3a9a62a02f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:54.423931 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec60d2a-1a1e-4213-81b6-3a9a62a02f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:54.426087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec60d2a-1a1e-4213-81b6-3a9a62a02f58 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:54.483981 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1403/2775] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:54 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:28:54.486391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d49c4-2277-429d-8ac1-3da0b94147cc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:54.499755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d49c4-2277-429d-8ac1-3da0b94147cc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:54.634610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055c469-2104-46b3-a0aa-a0d48ebbb8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:54.636099 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055c469-2104-46b3-a0aa-a0d48ebbb8ad admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:54.636562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055c469-2104-46b3-a0aa-a0d48ebbb8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:54.636994 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055c469-2104-46b3-a0aa-a0d48ebbb8ad admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:54.638507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055c469-2104-46b3-a0aa-a0d48ebbb8ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:54.640789 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055c469-2104-46b3-a0aa-a0d48ebbb8ad admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:54.680302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1373/2776] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:54 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:54.681364 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec60d2a-1a1e-4213-81b6-3a9a62a02f58 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:54.705560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec60d2a-1a1e-4213-81b6-3a9a62a02f58 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:54.711421 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e5eb4-ef02-4f2e-a887-17174f16f3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:54.712292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e5eb4-ef02-4f2e-a887-17174f16f3eb admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:54.712813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e5eb4-ef02-4f2e-a887-17174f16f3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:54.713247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e5eb4-ef02-4f2e-a887-17174f16f3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:54.715292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e5eb4-ef02-4f2e-a887-17174f16f3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:54.717937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e5eb4-ef02-4f2e-a887-17174f16f3eb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:54.743376 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1404/2777] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:54.747864 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055c469-2104-46b3-a0aa-a0d48ebbb8ad admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:54.763203 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055c469-2104-46b3-a0aa-a0d48ebbb8ad admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:54.775106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfffd31c-dd62-47ad-a465-3d2202b0794e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:54.775808 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffd31c-dd62-47ad-a465-3d2202b0794e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:54.777152 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffd31c-dd62-47ad-a465-3d2202b0794e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:54.777434 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffd31c-dd62-47ad-a465-3d2202b0794e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:54.778128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfffd31c-dd62-47ad-a465-3d2202b0794e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:54.780650 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfffd31c-dd62-47ad-a465-3d2202b0794e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:54.797661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1374/2778] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:54.801828 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e5eb4-ef02-4f2e-a887-17174f16f3eb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:54.818945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e5eb4-ef02-4f2e-a887-17174f16f3eb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:54.834344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c25ea3-4b01-4acd-8f56-b7c5353e9500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:54.835355 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c25ea3-4b01-4acd-8f56-b7c5353e9500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:54.835725 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c25ea3-4b01-4acd-8f56-b7c5353e9500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:54.836096 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c25ea3-4b01-4acd-8f56-b7c5353e9500 admin admin] PATH_INFO: `/v3/projects/d77ca289b79f4074b17d3333288108e5/users/779e791d332947bd88631e5e84b38669/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:54.863994 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c25ea3-4b01-4acd-8f56-b7c5353e9500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77ca289b79f4074b17d3333288108e5, user_id=779e791d332947bd88631e5e84b38669, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:54.867184 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c25ea3-4b01-4acd-8f56-b7c5353e9500 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:54.905063 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1405/2779] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:54 2022] PUT /identity/v3/projects/d77ca289b79f4074b17d3333288108e5/users/779e791d332947bd88631e5e84b38669/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:54.910920 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cb778-ee40-41d7-9224-8cf91db41f35 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:54.911233 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cb778-ee40-41d7-9224-8cf91db41f35 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:54.911491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cb778-ee40-41d7-9224-8cf91db41f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:54.967260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-040cb778-ee40-41d7-9224-8cf91db41f35 None None] MFA Rules not processed for user `779e791d332947bd88631e5e84b38669`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:54.968503 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-040cb778-ee40-41d7-9224-8cf91db41f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:54.993620 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:55.060260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1375/2780] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:55.072701 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c25ea3-4b01-4acd-8f56-b7c5353e9500 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:55.079338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-26c25ea3-4b01-4acd-8f56-b7c5353e9500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:55.190853 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c25ea3-4b01-4acd-8f56-b7c5353e9500 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:55.196026 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8007ea-461f-4f99-9ce1-8ea9c233c7f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:55.197142 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8007ea-461f-4f99-9ce1-8ea9c233c7f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:55.197593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8007ea-461f-4f99-9ce1-8ea9c233c7f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:55.199223 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8007ea-461f-4f99-9ce1-8ea9c233c7f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:55.215709 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8007ea-461f-4f99-9ce1-8ea9c233c7f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:55.217883 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8007ea-461f-4f99-9ce1-8ea9c233c7f1 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:55.232631 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1406/2781] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:28:55.449411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cb778-ee40-41d7-9224-8cf91db41f35 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:55.451854 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-040cb778-ee40-41d7-9224-8cf91db41f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:55.539147 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cb778-ee40-41d7-9224-8cf91db41f35 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:55.547180 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a87c00-f9b7-4b55-9987-e7f5225f4d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:55.548073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a87c00-f9b7-4b55-9987-e7f5225f4d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:55.548354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a87c00-f9b7-4b55-9987-e7f5225f4d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:55.548627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a87c00-f9b7-4b55-9987-e7f5225f4d44 admin admin] PATH_INFO: `/v3/users/b36eda4f44b9428d9ff648979059fb5b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:55.550024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a87c00-f9b7-4b55-9987-e7f5225f4d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36eda4f44b9428d9ff648979059fb5b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:55.552009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a87c00-f9b7-4b55-9987-e7f5225f4d44 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:55.636953 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a6a87c00-f9b7-4b55-9987-e7f5225f4d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36eda4f44b9428d9ff648979059fb5b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:55.650734 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a6a87c00-f9b7-4b55-9987-e7f5225f4d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36eda4f44b9428d9ff648979059fb5b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:55.652175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a6a87c00-f9b7-4b55-9987-e7f5225f4d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36eda4f44b9428d9ff648979059fb5b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:55.672072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a6a87c00-f9b7-4b55-9987-e7f5225f4d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36eda4f44b9428d9ff648979059fb5b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:55.696109 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a6a87c00-f9b7-4b55-9987-e7f5225f4d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36eda4f44b9428d9ff648979059fb5b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:55.707316 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1376/2782] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:28:55 2022] DELETE /identity/v3/users/b36eda4f44b9428d9ff648979059fb5b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:55.930707 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8007ea-461f-4f99-9ce1-8ea9c233c7f1 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:55.944592 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8007ea-461f-4f99-9ce1-8ea9c233c7f1 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:55.963196 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:56.027083 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:56.028318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.028582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.028856 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] PATH_INFO: `/v3/projects/1fe1de3697c04df4865c4a145a524231` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.030283 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe1de3697c04df4865c4a145a524231)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:56.032197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:56.102964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe1de3697c04df4865c4a145a524231', 'request_id': 'req-fa2bce67-9663-4f9c-b4bb-654978ba6598'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:56.116914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe1de3697c04df4865c4a145a524231', 'request_id': 'req-fa2bce67-9663-4f9c-b4bb-654978ba6598'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:56.130527 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe1de3697c04df4865c4a145a524231', 'request_id': 'req-fa2bce67-9663-4f9c-b4bb-654978ba6598'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:56.158365 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] The token cache is being invalidate because project 1fe1de3697c04df4865c4a145a524231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:28:56.158878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:56.161459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa2bce67-9663-4f9c-b4bb-654978ba6598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:28:56.164317 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1407/2783] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:28:55 2022] DELETE /identity/v3/projects/1fe1de3697c04df4865c4a145a524231 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:28:56.169326 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b508ed-2549-4f23-b88c-ba59287b7508 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.169723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b508ed-2549-4f23-b88c-ba59287b7508 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.170447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b508ed-2549-4f23-b88c-ba59287b7508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.191318 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d5b508ed-2549-4f23-b88c-ba59287b7508 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:56.193198 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d5b508ed-2549-4f23-b88c-ba59287b7508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:56.234885 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:56.280547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1377/2784] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:56.288688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5398eb-e250-45f0-ba3a-c32c4fe5cb40 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.289340 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5398eb-e250-45f0-ba3a-c32c4fe5cb40 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.290077 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5398eb-e250-45f0-ba3a-c32c4fe5cb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.331613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6e5398eb-e250-45f0-ba3a-c32c4fe5cb40 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:56.333062 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6e5398eb-e250-45f0-ba3a-c32c4fe5cb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:56.345344 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1408/2785] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:56.351547 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b508ed-2549-4f23-b88c-ba59287b7508 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:56.359044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c873954-f3a2-41fb-a3f7-91e5ea8f5cae None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.359338 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c873954-f3a2-41fb-a3f7-91e5ea8f5cae None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.359622 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c873954-f3a2-41fb-a3f7-91e5ea8f5cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.364612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b508ed-2549-4f23-b88c-ba59287b7508 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:56.369582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b15ac-22b1-42c6-af64-f822a4fbe3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:56.371551 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b15ac-22b1-42c6-af64-f822a4fbe3ce admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.372206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b15ac-22b1-42c6-af64-f822a4fbe3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.372705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b15ac-22b1-42c6-af64-f822a4fbe3ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.373759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b15ac-22b1-42c6-af64-f822a4fbe3ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:28:56.374356 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b15ac-22b1-42c6-af64-f822a4fbe3ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:56.376416 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b15ac-22b1-42c6-af64-f822a4fbe3ce admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:56.393426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7c873954-f3a2-41fb-a3f7-91e5ea8f5cae None None] MFA Rules not processed for user `a8fb914d98984987b928e79d4b75aa4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:56.394609 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1378/2786] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:28:56.396147 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7c873954-f3a2-41fb-a3f7-91e5ea8f5cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:56.399197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b15ac-22b1-42c6-af64-f822a4fbe3ce admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:56.412095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b15ac-22b1-42c6-af64-f822a4fbe3ce admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:56.419341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269060e1-eb72-4454-90d4-dcfb8439f482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:56.420615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269060e1-eb72-4454-90d4-dcfb8439f482 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.421425 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269060e1-eb72-4454-90d4-dcfb8439f482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.421958 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269060e1-eb72-4454-90d4-dcfb8439f482 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.423013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269060e1-eb72-4454-90d4-dcfb8439f482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:56.430508 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269060e1-eb72-4454-90d4-dcfb8439f482 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:56.469108 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1379/2787] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:28:56 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:28:56.472985 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269060e1-eb72-4454-90d4-dcfb8439f482 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:56.492249 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1409/2788] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:56.505983 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c873954-f3a2-41fb-a3f7-91e5ea8f5cae None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:56.508158 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7c873954-f3a2-41fb-a3f7-91e5ea8f5cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:56.554847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:56.572764 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269060e1-eb72-4454-90d4-dcfb8439f482 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:56.596476 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050236e-5660-4af2-976a-a75fa5258ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:56.597206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050236e-5660-4af2-976a-a75fa5258ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.597472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050236e-5660-4af2-976a-a75fa5258ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.597782 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050236e-5660-4af2-976a-a75fa5258ff1 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.598904 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050236e-5660-4af2-976a-a75fa5258ff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:56.602193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050236e-5660-4af2-976a-a75fa5258ff1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:56.622477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c873954-f3a2-41fb-a3f7-91e5ea8f5cae None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:56.634028 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1380/2789] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:28:56 2022] POST /identity/v3/users => generated 283 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:28:56.638311 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050236e-5660-4af2-976a-a75fa5258ff1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:56.656883 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050236e-5660-4af2-976a-a75fa5258ff1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:56.662811 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95cc72-8761-4db3-9a68-3c243bf4bcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:56.663757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95cc72-8761-4db3-9a68-3c243bf4bcab admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.664292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95cc72-8761-4db3-9a68-3c243bf4bcab admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.664798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95cc72-8761-4db3-9a68-3c243bf4bcab admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.665686 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95cc72-8761-4db3-9a68-3c243bf4bcab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:56.669681 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95cc72-8761-4db3-9a68-3c243bf4bcab admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:56.684208 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1381/2790] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:28:56.685292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95cc72-8761-4db3-9a68-3c243bf4bcab admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:56.697311 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95cc72-8761-4db3-9a68-3c243bf4bcab admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:56.703255 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8f785-f007-4aca-b52f-c2d36c53b225 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8386ca93fe48460ba5b6f0c10dc51122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:56.703921 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8f785-f007-4aca-b52f-c2d36c53b225 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.704207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8f785-f007-4aca-b52f-c2d36c53b225 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.704483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8f785-f007-4aca-b52f-c2d36c53b225 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.706526 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-33f8f785-f007-4aca-b52f-c2d36c53b225 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:56.710945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d07ff0-87cb-404f-a7ce-27965487de55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:56.712159 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d07ff0-87cb-404f-a7ce-27965487de55 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.712537 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d07ff0-87cb-404f-a7ce-27965487de55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.714240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d07ff0-87cb-404f-a7ce-27965487de55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.715059 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d07ff0-87cb-404f-a7ce-27965487de55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:56.716728 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d07ff0-87cb-404f-a7ce-27965487de55 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:56.729677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1382/2791] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:28:56.733648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d07ff0-87cb-404f-a7ce-27965487de55 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:56.747968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d07ff0-87cb-404f-a7ce-27965487de55 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:56.781879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d425c3-d1b8-4a1d-9b8d-c17b1f886329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:56.782795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d425c3-d1b8-4a1d-9b8d-c17b1f886329 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.783075 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d425c3-d1b8-4a1d-9b8d-c17b1f886329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.783380 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d425c3-d1b8-4a1d-9b8d-c17b1f886329 admin admin] PATH_INFO: `/v3/projects/a069cc0616cf4a0388529dd0a5071a2f/users/1284473ecd8e4957ad724b2d4318dfac/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.795616 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8f785-f007-4aca-b52f-c2d36c53b225 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:56.797159 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8f785-f007-4aca-b52f-c2d36c53b225 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:56.803595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d425c3-d1b8-4a1d-9b8d-c17b1f886329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a069cc0616cf4a0388529dd0a5071a2f, user_id=1284473ecd8e4957ad724b2d4318dfac, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:56.806477 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d425c3-d1b8-4a1d-9b8d-c17b1f886329 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:56.807035 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1410/2792] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:28:56.808979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65325d-200a-4cec-8f54-a75dea9ad570 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.809391 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65325d-200a-4cec-8f54-a75dea9ad570 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.809911 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65325d-200a-4cec-8f54-a75dea9ad570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.830748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8f65325d-200a-4cec-8f54-a75dea9ad570 None None] MFA Rules not processed for user `779e791d332947bd88631e5e84b38669`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:56.832085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8f65325d-200a-4cec-8f54-a75dea9ad570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:56.833285 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1383/2793] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:28:56 2022] PUT /identity/v3/projects/a069cc0616cf4a0388529dd0a5071a2f/users/1284473ecd8e4957ad724b2d4318dfac/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:28:56.843646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636771e-d2ed-4880-a7ec-23bcee7df209 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.844064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636771e-d2ed-4880-a7ec-23bcee7df209 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.844418 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636771e-d2ed-4880-a7ec-23bcee7df209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.895032 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e636771e-d2ed-4880-a7ec-23bcee7df209 None None] MFA Rules not processed for user `1284473ecd8e4957ad724b2d4318dfac`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:56.896281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e636771e-d2ed-4880-a7ec-23bcee7df209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:56.915274 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1411/2794] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:56.942797 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4e694-11be-436b-91e6-caebf1d7d6d1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:56.943206 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4e694-11be-436b-91e6-caebf1d7d6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:56.943766 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4e694-11be-436b-91e6-caebf1d7d6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:56.966772 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-93d4e694-11be-436b-91e6-caebf1d7d6d1 None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:28:56.968197 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-93d4e694-11be-436b-91e6-caebf1d7d6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:56.969543 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:57.012526 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1384/2795] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:28:57.024673 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636771e-d2ed-4880-a7ec-23bcee7df209 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:57.026836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e636771e-d2ed-4880-a7ec-23bcee7df209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:57.028972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:28:57.053671 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1412/2796] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:28:57.073676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4e694-11be-436b-91e6-caebf1d7d6d1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:28:57.076335 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-93d4e694-11be-436b-91e6-caebf1d7d6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:28:57.135615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636771e-d2ed-4880-a7ec-23bcee7df209 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:57.146990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7275857-4acc-41b7-bc50-df096890fc28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:57.148113 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7275857-4acc-41b7-bc50-df096890fc28 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:57.150607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7275857-4acc-41b7-bc50-df096890fc28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:57.151126 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7275857-4acc-41b7-bc50-df096890fc28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:57.161964 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7275857-4acc-41b7-bc50-df096890fc28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:57.163383 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7275857-4acc-41b7-bc50-df096890fc28 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:57.176640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1385/2797] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:28:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:28:57.179457 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4e694-11be-436b-91e6-caebf1d7d6d1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:28:57.189449 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021c5ac-1d9a-40d3-ad00-c2f7957e20df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:28:57.190894 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021c5ac-1d9a-40d3-ad00-c2f7957e20df service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:28:57.191191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021c5ac-1d9a-40d3-ad00-c2f7957e20df service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:28:57.191683 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021c5ac-1d9a-40d3-ad00-c2f7957e20df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:28:57.209317 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021c5ac-1d9a-40d3-ad00-c2f7957e20df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:28:57.212490 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021c5ac-1d9a-40d3-ad00-c2f7957e20df service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:28:57.253640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1413/2798] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:28:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:29:07.171481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=36 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:07.248987 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=38 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:07.967021 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24e9b-bf85-4d0d-bbca-a14c5ea8519f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:07.967335 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24e9b-bf85-4d0d-bbca-a14c5ea8519f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:07.967612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24e9b-bf85-4d0d-bbca-a14c5ea8519f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:07.985135 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-eda24e9b-bf85-4d0d-bbca-a14c5ea8519f None None] MFA Rules not processed for user `fcca87cd01ae464cb1484c3bbd52aaf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:07.986351 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-eda24e9b-bf85-4d0d-bbca-a14c5ea8519f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:08.044495 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1386/2799] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:10.555424 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5baff4f-04d9-45d7-af56-a60246d413dd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:10.555941 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5baff4f-04d9-45d7-af56-a60246d413dd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:10.556067 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5baff4f-04d9-45d7-af56-a60246d413dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:10.575668 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d5baff4f-04d9-45d7-af56-a60246d413dd None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:10.576867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d5baff4f-04d9-45d7-af56-a60246d413dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:10.584433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1414/2800] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:29:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:10.597415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda24e9b-bf85-4d0d-bbca-a14c5ea8519f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:10.609334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda24e9b-bf85-4d0d-bbca-a14c5ea8519f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:10.614759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fc39a-8be0-465a-9618-f3911a98ccae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:10.615406 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fc39a-8be0-465a-9618-f3911a98ccae service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:10.615687 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fc39a-8be0-465a-9618-f3911a98ccae service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:10.615955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fc39a-8be0-465a-9618-f3911a98ccae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:10.625260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951fc39a-8be0-465a-9618-f3911a98ccae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:10.626861 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951fc39a-8be0-465a-9618-f3911a98ccae service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:10.634929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1387/2801] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:29:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:29:20.570045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:20.631576 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=31 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:25.974247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3432d9-6a7c-4d96-8534-7f8a33447c6a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:25.974747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3432d9-6a7c-4d96-8534-7f8a33447c6a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:25.974862 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3432d9-6a7c-4d96-8534-7f8a33447c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:25.992212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cf3432d9-6a7c-4d96-8534-7f8a33447c6a None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:25.993554 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cf3432d9-6a7c-4d96-8534-7f8a33447c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:26.001099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1415/2802] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:29:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:26.013376 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fc39a-8be0-465a-9618-f3911a98ccae service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:26.023837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fc39a-8be0-465a-9618-f3911a98ccae service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:26.029492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd40fd-5406-4ed1-b2f9-39af8a1e226e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:26.030164 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd40fd-5406-4ed1-b2f9-39af8a1e226e service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:26.030467 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd40fd-5406-4ed1-b2f9-39af8a1e226e service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:26.030743 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd40fd-5406-4ed1-b2f9-39af8a1e226e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:26.039187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd40fd-5406-4ed1-b2f9-39af8a1e226e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:26.040699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd40fd-5406-4ed1-b2f9-39af8a1e226e service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:26.049647 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1388/2803] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:29:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:29:26.614862 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11046e07-3c1e-41d6-a194-05b354495170 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:26.615244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11046e07-3c1e-41d6-a194-05b354495170 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:26.615625 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11046e07-3c1e-41d6-a194-05b354495170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:26.641455 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-11046e07-3c1e-41d6-a194-05b354495170 None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:26.642845 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-11046e07-3c1e-41d6-a194-05b354495170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:26.652775 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1416/2804] 10.222.0.42 () {58 vars in 967 bytes} [Wed May 25 16:29:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:29:26.669363 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd40fd-5406-4ed1-b2f9-39af8a1e226e service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:26.680101 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd40fd-5406-4ed1-b2f9-39af8a1e226e service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:26.685803 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212aefd-9afe-4f32-b5b0-c2425c2687a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:26.686460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212aefd-9afe-4f32-b5b0-c2425c2687a6 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:26.687161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212aefd-9afe-4f32-b5b0-c2425c2687a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:26.687161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212aefd-9afe-4f32-b5b0-c2425c2687a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:26.695981 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212aefd-9afe-4f32-b5b0-c2425c2687a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:26.697646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212aefd-9afe-4f32-b5b0-c2425c2687a6 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:26.707757 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1389/2805] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:29:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:29:26.911861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195b1ed-794b-449e-b86b-08adc1052cff None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:26.911861 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195b1ed-794b-449e-b86b-08adc1052cff None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:26.912155 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195b1ed-794b-449e-b86b-08adc1052cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:26.932133 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d195b1ed-794b-449e-b86b-08adc1052cff None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:26.933903 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d195b1ed-794b-449e-b86b-08adc1052cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:26.942148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1417/2806] 10.222.0.42 () {58 vars in 1009 bytes} [Wed May 25 16:29:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:26.957451 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212aefd-9afe-4f32-b5b0-c2425c2687a6 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:26.969329 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212aefd-9afe-4f32-b5b0-c2425c2687a6 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:26.974387 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f38355-0fbc-40da-bb36-726049bf047d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:26.975020 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f38355-0fbc-40da-bb36-726049bf047d service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:26.975328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f38355-0fbc-40da-bb36-726049bf047d service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:26.975598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f38355-0fbc-40da-bb36-726049bf047d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:26.984939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f38355-0fbc-40da-bb36-726049bf047d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:26.986555 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f38355-0fbc-40da-bb36-726049bf047d service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:26.994615 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1390/2807] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:29:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:29:27.309414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ed921-e44d-4857-b000-4e466acf12f7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:27.309926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ed921-e44d-4857-b000-4e466acf12f7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:27.310123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ed921-e44d-4857-b000-4e466acf12f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:27.327532 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-833ed921-e44d-4857-b000-4e466acf12f7 None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:27.328768 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-833ed921-e44d-4857-b000-4e466acf12f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:27.335815 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1418/2808] 10.222.0.42 () {58 vars in 967 bytes} [Wed May 25 16:29:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:27.347527 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f38355-0fbc-40da-bb36-726049bf047d service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:27.358445 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f38355-0fbc-40da-bb36-726049bf047d service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:27.363533 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43baa330-9195-4b9b-b171-36718b7ca558 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:27.364165 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43baa330-9195-4b9b-b171-36718b7ca558 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:27.364450 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43baa330-9195-4b9b-b171-36718b7ca558 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:27.364754 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43baa330-9195-4b9b-b171-36718b7ca558 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:27.373562 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43baa330-9195-4b9b-b171-36718b7ca558 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:27.375188 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43baa330-9195-4b9b-b171-36718b7ca558 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:27.383071 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1391/2809] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:29:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:29:29.655530 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ed921-e44d-4857-b000-4e466acf12f7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:29.656937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-833ed921-e44d-4857-b000-4e466acf12f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:29.726553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ed921-e44d-4857-b000-4e466acf12f7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:29.731123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:29.732005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:29.732292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:29.732568 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] PATH_INFO: `/v3/users/779e791d332947bd88631e5e84b38669` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:29.733822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=779e791d332947bd88631e5e84b38669)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:29.735762 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:29.783363 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779e791d332947bd88631e5e84b38669'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:29.784068 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779e791d332947bd88631e5e84b38669'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:29.784708 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779e791d332947bd88631e5e84b38669'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:29.797001 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779e791d332947bd88631e5e84b38669'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:29.803786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779e791d332947bd88631e5e84b38669'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:29.817217 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1419/2810] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:29:29 2022] DELETE /identity/v3/users/779e791d332947bd88631e5e84b38669 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:29:29.821677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3de9a-60b1-42c1-80de-966f0360b5c9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:29.822181 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3de9a-60b1-42c1-80de-966f0360b5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:29.822288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3de9a-60b1-42c1-80de-966f0360b5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:29.841875 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9fd3de9a-60b1-42c1-80de-966f0360b5c9 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:29.843079 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9fd3de9a-60b1-42c1-80de-966f0360b5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:29.899726 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1392/2811] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:29.909926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:29.911324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:29.969224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67685548-12cb-4017-ad52-4ed163522ee3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:29.972961 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e40ff5-f953-4ec4-8eb5-751c267e82da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:29.973617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e40ff5-f953-4ec4-8eb5-751c267e82da service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:29.973932 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e40ff5-f953-4ec4-8eb5-751c267e82da service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:29.974212 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e40ff5-f953-4ec4-8eb5-751c267e82da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:29.986782 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:29.989675 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e40ff5-f953-4ec4-8eb5-751c267e82da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:29.991284 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e40ff5-f953-4ec4-8eb5-751c267e82da service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:30.003667 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1420/2812] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:29:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:29:30.176949 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3de9a-60b1-42c1-80de-966f0360b5c9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:30.178571 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9fd3de9a-60b1-42c1-80de-966f0360b5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:30.191319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3de9a-60b1-42c1-80de-966f0360b5c9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:30.197968 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:30.199767 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:30.200133 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:30.200323 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] PATH_INFO: `/v3/projects/d77ca289b79f4074b17d3333288108e5` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:30.201610 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d77ca289b79f4074b17d3333288108e5)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:30.203616 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:30.278491 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd77ca289b79f4074b17d3333288108e5', 'request_id': 'req-ed975558-b3e1-437a-8102-25341eddf32a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:30.285391 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd77ca289b79f4074b17d3333288108e5', 'request_id': 'req-ed975558-b3e1-437a-8102-25341eddf32a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:30.292401 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd77ca289b79f4074b17d3333288108e5', 'request_id': 'req-ed975558-b3e1-437a-8102-25341eddf32a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:30.308415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] The token cache is being invalidate because project d77ca289b79f4074b17d3333288108e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:29:30.308960 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:30.309876 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ed975558-b3e1-437a-8102-25341eddf32a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:30.312219 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1393/2813] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:29:30 2022] DELETE /identity/v3/projects/d77ca289b79f4074b17d3333288108e5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:29:30.316956 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc482b19-0299-4da6-bd17-4bceab6595b5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:30.317248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc482b19-0299-4da6-bd17-4bceab6595b5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:30.317520 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc482b19-0299-4da6-bd17-4bceab6595b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:30.337590 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dc482b19-0299-4da6-bd17-4bceab6595b5 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:30.338965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dc482b19-0299-4da6-bd17-4bceab6595b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:30.413748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1421/2814] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:30.477372 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad024-c595-42bb-8463-22adca84d999 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:30.477646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad024-c595-42bb-8463-22adca84d999 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:30.477961 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad024-c595-42bb-8463-22adca84d999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:30.497286 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-65dad024-c595-42bb-8463-22adca84d999 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:30.498466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65dad024-c595-42bb-8463-22adca84d999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:30.506481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1394/2815] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:30.510854 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc482b19-0299-4da6-bd17-4bceab6595b5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:30.521232 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc482b19-0299-4da6-bd17-4bceab6595b5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:30.527066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8d3ca-15f5-4095-9b3e-40b763be691c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:30.527916 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8d3ca-15f5-4095-9b3e-40b763be691c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:30.528209 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8d3ca-15f5-4095-9b3e-40b763be691c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:30.528706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8d3ca-15f5-4095-9b3e-40b763be691c admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:30.529442 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8d3ca-15f5-4095-9b3e-40b763be691c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:29:30.529777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8d3ca-15f5-4095-9b3e-40b763be691c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:30.531930 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8d3ca-15f5-4095-9b3e-40b763be691c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:30.545164 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1422/2816] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:29:30.549700 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dad024-c595-42bb-8463-22adca84d999 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:30.559779 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dad024-c595-42bb-8463-22adca84d999 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:30.564517 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860ad6e-5886-4a8e-8d33-1fdbd56056bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:30.565171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860ad6e-5886-4a8e-8d33-1fdbd56056bb admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:30.565483 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860ad6e-5886-4a8e-8d33-1fdbd56056bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:30.565919 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860ad6e-5886-4a8e-8d33-1fdbd56056bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:30.566948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860ad6e-5886-4a8e-8d33-1fdbd56056bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:30.569790 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860ad6e-5886-4a8e-8d33-1fdbd56056bb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:30.600780 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1395/2817] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:29:30 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:29:30.601979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8d3ca-15f5-4095-9b3e-40b763be691c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:30.612191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8d3ca-15f5-4095-9b3e-40b763be691c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:30.674436 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8470be9-01dc-419f-9712-86b627d23b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:30.675106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470be9-01dc-419f-9712-86b627d23b29 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:30.675382 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470be9-01dc-419f-9712-86b627d23b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:30.675659 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470be9-01dc-419f-9712-86b627d23b29 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:30.676381 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8470be9-01dc-419f-9712-86b627d23b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:30.678342 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8470be9-01dc-419f-9712-86b627d23b29 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:30.699257 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1423/2818] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:29:30 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:29:30.703639 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860ad6e-5886-4a8e-8d33-1fdbd56056bb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:30.709003 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:30.715677 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860ad6e-5886-4a8e-8d33-1fdbd56056bb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:30.721285 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fbf46-bbee-4fa1-a3fd-9100ab010123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:30.721975 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fbf46-bbee-4fa1-a3fd-9100ab010123 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:30.722257 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fbf46-bbee-4fa1-a3fd-9100ab010123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:30.722529 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fbf46-bbee-4fa1-a3fd-9100ab010123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:30.723242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69fbf46-bbee-4fa1-a3fd-9100ab010123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:30.724815 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69fbf46-bbee-4fa1-a3fd-9100ab010123 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:30.735250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1396/2819] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:29:30.739157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8470be9-01dc-419f-9712-86b627d23b29 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:30.750105 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8470be9-01dc-419f-9712-86b627d23b29 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:30.754640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe8501-d4d0-475d-908d-be6124f4870e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:30.755290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe8501-d4d0-475d-908d-be6124f4870e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:30.755570 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe8501-d4d0-475d-908d-be6124f4870e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:30.755846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe8501-d4d0-475d-908d-be6124f4870e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:30.756566 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe8501-d4d0-475d-908d-be6124f4870e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:30.758199 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe8501-d4d0-475d-908d-be6124f4870e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:30.769014 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1424/2820] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:29:30.773203 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fbf46-bbee-4fa1-a3fd-9100ab010123 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:30.783386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fbf46-bbee-4fa1-a3fd-9100ab010123 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:30.788193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a663c8-5cfd-488f-9b3e-860e839024f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:30.789055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a663c8-5cfd-488f-9b3e-860e839024f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:30.789332 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a663c8-5cfd-488f-9b3e-860e839024f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:30.789604 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a663c8-5cfd-488f-9b3e-860e839024f3 admin admin] PATH_INFO: `/v3/projects/9f6a5a6ae8b2488f8a5af6d2066fa88e/users/28d0cf08accf4a28b85ac2342d9c3023/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:30.803801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a663c8-5cfd-488f-9b3e-860e839024f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f6a5a6ae8b2488f8a5af6d2066fa88e, user_id=28d0cf08accf4a28b85ac2342d9c3023, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:30.806480 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a663c8-5cfd-488f-9b3e-860e839024f3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:30.820494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1397/2821] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:29:30 2022] PUT /identity/v3/projects/9f6a5a6ae8b2488f8a5af6d2066fa88e/users/28d0cf08accf4a28b85ac2342d9c3023/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:29:30.826354 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5452b-59dd-4436-8cbd-279fb4afdc77 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:30.826650 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5452b-59dd-4436-8cbd-279fb4afdc77 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:30.826935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5452b-59dd-4436-8cbd-279fb4afdc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:30.844478 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-91e5452b-59dd-4436-8cbd-279fb4afdc77 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:30.845681 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-91e5452b-59dd-4436-8cbd-279fb4afdc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:30.903004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1425/2822] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:29:30.912902 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a663c8-5cfd-488f-9b3e-860e839024f3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:30.914683 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-55a663c8-5cfd-488f-9b3e-860e839024f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:30.976990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a663c8-5cfd-488f-9b3e-860e839024f3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:30.981464 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572332b6-32c2-4378-baf0-86653370ed51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:30.982150 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572332b6-32c2-4378-baf0-86653370ed51 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:30.982438 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572332b6-32c2-4378-baf0-86653370ed51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:30.982722 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572332b6-32c2-4378-baf0-86653370ed51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:30.991587 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572332b6-32c2-4378-baf0-86653370ed51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:30.993014 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572332b6-32c2-4378-baf0-86653370ed51 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:31.003081 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1398/2823] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:29:31.684431 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead9507-ad5e-473a-bdbf-bf02e49e67d2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:31.684917 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead9507-ad5e-473a-bdbf-bf02e49e67d2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:31.685004 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead9507-ad5e-473a-bdbf-bf02e49e67d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:31.692162 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:31.720827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5ead9507-ad5e-473a-bdbf-bf02e49e67d2 None None] MFA Rules not processed for user `28d0cf08accf4a28b85ac2342d9c3023`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:31.722094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5ead9507-ad5e-473a-bdbf-bf02e49e67d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:31.810438 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1426/2824] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:31.821282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572332b6-32c2-4378-baf0-86653370ed51 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:31.822871 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-572332b6-32c2-4378-baf0-86653370ed51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:31.896338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572332b6-32c2-4378-baf0-86653370ed51 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:31.904288 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84ce23-ef2f-4c4b-84ab-59a0c3116470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:31.905119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84ce23-ef2f-4c4b-84ab-59a0c3116470 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:31.905422 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84ce23-ef2f-4c4b-84ab-59a0c3116470 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:31.905841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84ce23-ef2f-4c4b-84ab-59a0c3116470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:31.918623 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84ce23-ef2f-4c4b-84ab-59a0c3116470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:31.920853 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84ce23-ef2f-4c4b-84ab-59a0c3116470 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:31.934574 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1399/2825] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:29:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:29:33.307967 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcc4b7-8e58-477c-9fa5-aed7ab34182e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:33.308479 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcc4b7-8e58-477c-9fa5-aed7ab34182e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:33.308576 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcc4b7-8e58-477c-9fa5-aed7ab34182e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:33.326688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2ebcc4b7-8e58-477c-9fa5-aed7ab34182e None None] MFA Rules not processed for user `28d0cf08accf4a28b85ac2342d9c3023`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:33.327878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2ebcc4b7-8e58-477c-9fa5-aed7ab34182e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:33.334975 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1427/2826] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:33.771748 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84ce23-ef2f-4c4b-84ab-59a0c3116470 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:33.774000 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8d84ce23-ef2f-4c4b-84ab-59a0c3116470 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:33.791685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84ce23-ef2f-4c4b-84ab-59a0c3116470 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:33.798184 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:33.799266 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:33.799725 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:33.800161 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] PATH_INFO: `/v3/users/1284473ecd8e4957ad724b2d4318dfac` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:33.801798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1284473ecd8e4957ad724b2d4318dfac)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:33.804123 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:33.876339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1284473ecd8e4957ad724b2d4318dfac'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:33.886895 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1284473ecd8e4957ad724b2d4318dfac'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:33.887638 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1284473ecd8e4957ad724b2d4318dfac'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:33.921940 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:33.925151 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1284473ecd8e4957ad724b2d4318dfac'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:33.940947 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1284473ecd8e4957ad724b2d4318dfac'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:33.966437 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1400/2827] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:29:33 2022] DELETE /identity/v3/users/1284473ecd8e4957ad724b2d4318dfac => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:29:33.968604 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d7fae-7863-4b3f-85c9-5f60e1427235 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:33.969087 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d7fae-7863-4b3f-85c9-5f60e1427235 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:33.969680 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d7fae-7863-4b3f-85c9-5f60e1427235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:33.993144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2a4d7fae-7863-4b3f-85c9-5f60e1427235 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:33.994613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2a4d7fae-7863-4b3f-85c9-5f60e1427235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:34.070648 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1428/2828] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:34.080892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:34.082723 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:34.148637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61860a54-1240-4c80-91d4-303f05c0dba3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:34.153532 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2686cc-eda4-47f1-86fc-f69eaf25446b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:34.156283 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2686cc-eda4-47f1-86fc-f69eaf25446b service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:34.157242 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2686cc-eda4-47f1-86fc-f69eaf25446b service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:34.157736 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2686cc-eda4-47f1-86fc-f69eaf25446b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:34.170625 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2686cc-eda4-47f1-86fc-f69eaf25446b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:34.172188 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2686cc-eda4-47f1-86fc-f69eaf25446b service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:34.182292 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1401/2829] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:29:34.490300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d7fae-7863-4b3f-85c9-5f60e1427235 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:34.492033 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2a4d7fae-7863-4b3f-85c9-5f60e1427235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:34.504831 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d7fae-7863-4b3f-85c9-5f60e1427235 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:34.510525 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:34.511527 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:34.511876 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:34.512226 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] PATH_INFO: `/v3/projects/a069cc0616cf4a0388529dd0a5071a2f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:34.513697 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a069cc0616cf4a0388529dd0a5071a2f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:34.515990 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:34.567309 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a069cc0616cf4a0388529dd0a5071a2f', 'request_id': 'req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:34.573461 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a069cc0616cf4a0388529dd0a5071a2f', 'request_id': 'req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:34.581246 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a069cc0616cf4a0388529dd0a5071a2f', 'request_id': 'req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:34.593385 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] The token cache is being invalidate because project a069cc0616cf4a0388529dd0a5071a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:29:34.593747 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:34.594495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3f3f04dc-c88a-45e4-b7e0-a14d6decf007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:34.596071 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1429/2830] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:29:34 2022] DELETE /identity/v3/projects/a069cc0616cf4a0388529dd0a5071a2f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:29:34.600929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f901f3-57f5-4b9a-a189-f9431700db04 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:34.601218 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f901f3-57f5-4b9a-a189-f9431700db04 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:34.601496 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f901f3-57f5-4b9a-a189-f9431700db04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:34.619038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d2f901f3-57f5-4b9a-a189-f9431700db04 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:34.620168 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d2f901f3-57f5-4b9a-a189-f9431700db04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:34.679564 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1402/2831] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:34.687244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41a5d5-e7fd-410e-aa22-8a8cac4332cb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:34.687541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41a5d5-e7fd-410e-aa22-8a8cac4332cb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:34.687822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41a5d5-e7fd-410e-aa22-8a8cac4332cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:34.706573 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bd41a5d5-e7fd-410e-aa22-8a8cac4332cb None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:34.707761 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bd41a5d5-e7fd-410e-aa22-8a8cac4332cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:34.715597 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1430/2832] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:34.720540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f901f3-57f5-4b9a-a189-f9431700db04 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:34.730049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f901f3-57f5-4b9a-a189-f9431700db04 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:34.734626 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf08aa-6da1-4969-9117-f18eb6434c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:34.735271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf08aa-6da1-4969-9117-f18eb6434c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:34.735617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf08aa-6da1-4969-9117-f18eb6434c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:34.735902 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf08aa-6da1-4969-9117-f18eb6434c26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:34.736619 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf08aa-6da1-4969-9117-f18eb6434c26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:29:34.737056 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf08aa-6da1-4969-9117-f18eb6434c26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:34.738662 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf08aa-6da1-4969-9117-f18eb6434c26 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:34.750766 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1403/2833] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:29:34.755189 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41a5d5-e7fd-410e-aa22-8a8cac4332cb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:34.759642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:34.766492 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41a5d5-e7fd-410e-aa22-8a8cac4332cb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:34.773240 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b26a33-d0cc-4f61-970d-bfb5393492e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:34.774224 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b26a33-d0cc-4f61-970d-bfb5393492e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:34.774515 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b26a33-d0cc-4f61-970d-bfb5393492e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:34.774899 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b26a33-d0cc-4f61-970d-bfb5393492e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:34.775800 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b26a33-d0cc-4f61-970d-bfb5393492e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:34.777746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b26a33-d0cc-4f61-970d-bfb5393492e4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:34.805319 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1431/2834] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:29:34 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:29:34.809488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf08aa-6da1-4969-9117-f18eb6434c26 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:34.821009 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf08aa-6da1-4969-9117-f18eb6434c26 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:34.894933 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a803550-c625-4cbf-8cee-c5af26402799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:34.895627 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a803550-c625-4cbf-8cee-c5af26402799 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:34.895915 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a803550-c625-4cbf-8cee-c5af26402799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:34.896193 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a803550-c625-4cbf-8cee-c5af26402799 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:34.897197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a803550-c625-4cbf-8cee-c5af26402799 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:34.901085 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a803550-c625-4cbf-8cee-c5af26402799 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:34.929507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1404/2835] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:29:34 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:29:34.933670 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b26a33-d0cc-4f61-970d-bfb5393492e4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:34.946640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b26a33-d0cc-4f61-970d-bfb5393492e4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:34.960248 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed3d9e-5ac1-446c-aefb-0faa6ba57894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:34.960938 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed3d9e-5ac1-446c-aefb-0faa6ba57894 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:34.961220 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed3d9e-5ac1-446c-aefb-0faa6ba57894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:34.961553 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed3d9e-5ac1-446c-aefb-0faa6ba57894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:34.962343 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed3d9e-5ac1-446c-aefb-0faa6ba57894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:34.964065 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed3d9e-5ac1-446c-aefb-0faa6ba57894 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:34.978200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1432/2836] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:29:34.983663 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a803550-c625-4cbf-8cee-c5af26402799 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:35.001359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:35.006948 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a803550-c625-4cbf-8cee-c5af26402799 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:35.026879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aaba30-827e-4ea4-a00b-013af6d4c367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:35.027594 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaba30-827e-4ea4-a00b-013af6d4c367 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:35.028093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaba30-827e-4ea4-a00b-013af6d4c367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:35.028397 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaba30-827e-4ea4-a00b-013af6d4c367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:35.029158 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aaba30-827e-4ea4-a00b-013af6d4c367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:35.039213 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aaba30-827e-4ea4-a00b-013af6d4c367 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:35.073980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1405/2837] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:29:35.074106 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed3d9e-5ac1-446c-aefb-0faa6ba57894 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:35.088425 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed3d9e-5ac1-446c-aefb-0faa6ba57894 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:35.102637 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624e5ef-1748-4fc2-880a-dc3769b58086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:35.104292 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624e5ef-1748-4fc2-880a-dc3769b58086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:35.105742 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624e5ef-1748-4fc2-880a-dc3769b58086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:35.108721 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624e5ef-1748-4fc2-880a-dc3769b58086 admin admin] PATH_INFO: `/v3/projects/cde664fec2774caf9abc571b5ab99b4c/users/cdd89049d6134137be5d0180ca527756/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:35.128999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624e5ef-1748-4fc2-880a-dc3769b58086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde664fec2774caf9abc571b5ab99b4c, user_id=cdd89049d6134137be5d0180ca527756, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:35.131324 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624e5ef-1748-4fc2-880a-dc3769b58086 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:35.158459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1433/2838] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:29:35 2022] PUT /identity/v3/projects/cde664fec2774caf9abc571b5ab99b4c/users/cdd89049d6134137be5d0180ca527756/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:29:35.171190 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d19d9-0cdf-4d0f-b991-360f6a94e9f2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:35.171481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d19d9-0cdf-4d0f-b991-360f6a94e9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:35.171759 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d19d9-0cdf-4d0f-b991-360f6a94e9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:35.203805 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-783d19d9-0cdf-4d0f-b991-360f6a94e9f2 None None] MFA Rules not processed for user `cdd89049d6134137be5d0180ca527756`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:35.205263 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-783d19d9-0cdf-4d0f-b991-360f6a94e9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:35.285381 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1406/2839] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:29:35.300122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624e5ef-1748-4fc2-880a-dc3769b58086 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:35.301952 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0624e5ef-1748-4fc2-880a-dc3769b58086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:35.364320 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624e5ef-1748-4fc2-880a-dc3769b58086 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:35.368144 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce4f51-b810-404e-a5db-92111f4e8f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9913b9eb502434985f84c4a11632ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:35.368860 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4f51-b810-404e-a5db-92111f4e8f1e service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:35.369207 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4f51-b810-404e-a5db-92111f4e8f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:35.369488 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4f51-b810-404e-a5db-92111f4e8f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:35.379152 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce4f51-b810-404e-a5db-92111f4e8f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:35.380608 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce4f51-b810-404e-a5db-92111f4e8f1e service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:35.390098 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1434/2840] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:29:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:29:35.416234 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d19d9-0cdf-4d0f-b991-360f6a94e9f2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:35.485131 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d19d9-0cdf-4d0f-b991-360f6a94e9f2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:35.489806 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:35.490682 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:35.490967 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:35.491244 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] PATH_INFO: `/v3/users/cdd89049d6134137be5d0180ca527756` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:35.492770 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdd89049d6134137be5d0180ca527756)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:35.494447 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:35.551070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdd89049d6134137be5d0180ca527756'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:35.559260 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdd89049d6134137be5d0180ca527756'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:35.560055 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdd89049d6134137be5d0180ca527756'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:35.574342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdd89049d6134137be5d0180ca527756'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:35.581475 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdd89049d6134137be5d0180ca527756'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:35.588346 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1407/2841] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:29:35 2022] DELETE /identity/v3/users/cdd89049d6134137be5d0180ca527756 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:29:35.593392 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb17e7d-cae4-44c8-85c3-91fd710efbe9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:35.593882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb17e7d-cae4-44c8-85c3-91fd710efbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:35.594045 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb17e7d-cae4-44c8-85c3-91fd710efbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:35.612006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7cb17e7d-cae4-44c8-85c3-91fd710efbe9 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:35.613215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7cb17e7d-cae4-44c8-85c3-91fd710efbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:35.674613 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1435/2842] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:35.684204 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:35.685667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:35.691223 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:35.748912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea9b99-ae35-492d-80cf-bd42ac356fc1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:35.752839 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83359d7-6c77-47f0-a957-8d45f7d3969a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:35.753505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83359d7-6c77-47f0-a957-8d45f7d3969a service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:35.753851 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83359d7-6c77-47f0-a957-8d45f7d3969a service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:35.754130 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83359d7-6c77-47f0-a957-8d45f7d3969a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:35.764057 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83359d7-6c77-47f0-a957-8d45f7d3969a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:35.765539 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83359d7-6c77-47f0-a957-8d45f7d3969a service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:35.775380 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1408/2843] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:29:36.061143 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb17e7d-cae4-44c8-85c3-91fd710efbe9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:36.062688 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7cb17e7d-cae4-44c8-85c3-91fd710efbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:36.075454 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb17e7d-cae4-44c8-85c3-91fd710efbe9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:36.084408 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:36.085433 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:36.085749 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:36.086047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] PATH_INFO: `/v3/projects/cde664fec2774caf9abc571b5ab99b4c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:36.087781 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde664fec2774caf9abc571b5ab99b4c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:36.096640 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:36.156483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:36.162094 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde664fec2774caf9abc571b5ab99b4c', 'request_id': 'req-e3f738ce-0693-414b-b7c6-0145c6a6f52c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:36.170558 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde664fec2774caf9abc571b5ab99b4c', 'request_id': 'req-e3f738ce-0693-414b-b7c6-0145c6a6f52c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:36.178502 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde664fec2774caf9abc571b5ab99b4c', 'request_id': 'req-e3f738ce-0693-414b-b7c6-0145c6a6f52c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:36.194256 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] The token cache is being invalidate because project cde664fec2774caf9abc571b5ab99b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:29:36.194960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:36.195872 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e3f738ce-0693-414b-b7c6-0145c6a6f52c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:29:36.197755 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1436/2844] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:29:36 2022] DELETE /identity/v3/projects/cde664fec2774caf9abc571b5ab99b4c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:29:36.203908 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d3562-80e9-4a8e-9549-5659eb8d0e98 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:36.204430 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d3562-80e9-4a8e-9549-5659eb8d0e98 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:36.204685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d3562-80e9-4a8e-9549-5659eb8d0e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:36.224084 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fd8d3562-80e9-4a8e-9549-5659eb8d0e98 None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:36.227211 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fd8d3562-80e9-4a8e-9549-5659eb8d0e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:36.332064 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1409/2845] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:36.340211 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87c70a-b1f7-4155-999a-d3c1524e8f8d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:36.340498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87c70a-b1f7-4155-999a-d3c1524e8f8d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:36.340785 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87c70a-b1f7-4155-999a-d3c1524e8f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:36.369146 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8a87c70a-b1f7-4155-999a-d3c1524e8f8d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:36.370323 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8a87c70a-b1f7-4155-999a-d3c1524e8f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:36.379302 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1437/2846] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:36.383515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d3562-80e9-4a8e-9549-5659eb8d0e98 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:36.396440 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d3562-80e9-4a8e-9549-5659eb8d0e98 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:36.401216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043252d6-4712-4684-a20c-35e33a7132f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:36.402053 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043252d6-4712-4684-a20c-35e33a7132f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:36.402341 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043252d6-4712-4684-a20c-35e33a7132f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:36.402713 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043252d6-4712-4684-a20c-35e33a7132f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:36.403458 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043252d6-4712-4684-a20c-35e33a7132f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:29:36.403775 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043252d6-4712-4684-a20c-35e33a7132f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:36.406802 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043252d6-4712-4684-a20c-35e33a7132f5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:36.420402 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1410/2847] 10.222.0.42 () {58 vars in 1152 bytes} [Wed May 25 16:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:29:36.424569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87c70a-b1f7-4155-999a-d3c1524e8f8d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:36.437141 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87c70a-b1f7-4155-999a-d3c1524e8f8d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:36.441906 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c233f1-92ba-4381-af2a-8f273e527a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:36.442564 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c233f1-92ba-4381-af2a-8f273e527a50 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:36.442846 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c233f1-92ba-4381-af2a-8f273e527a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:36.443356 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c233f1-92ba-4381-af2a-8f273e527a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:36.444048 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c233f1-92ba-4381-af2a-8f273e527a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:36.445972 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c233f1-92ba-4381-af2a-8f273e527a50 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:36.473311 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1438/2848] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:29:36 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:29:36.477824 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043252d6-4712-4684-a20c-35e33a7132f5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:36.488751 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043252d6-4712-4684-a20c-35e33a7132f5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:36.546913 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2ba8a-4094-4137-b6e3-04d50114928b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:36.548128 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2ba8a-4094-4137-b6e3-04d50114928b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:36.548380 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2ba8a-4094-4137-b6e3-04d50114928b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:36.548655 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2ba8a-4094-4137-b6e3-04d50114928b admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:36.549338 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2ba8a-4094-4137-b6e3-04d50114928b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:36.551890 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2ba8a-4094-4137-b6e3-04d50114928b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:36.574307 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1411/2849] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:29:36 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:29:36.578588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c233f1-92ba-4381-af2a-8f273e527a50 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:36.588901 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c233f1-92ba-4381-af2a-8f273e527a50 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:36.594170 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267777e3-95cb-420a-a315-6729dfbc08ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:36.594838 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267777e3-95cb-420a-a315-6729dfbc08ef admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:36.595122 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267777e3-95cb-420a-a315-6729dfbc08ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:36.595398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267777e3-95cb-420a-a315-6729dfbc08ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:36.596124 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267777e3-95cb-420a-a315-6729dfbc08ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:36.597748 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267777e3-95cb-420a-a315-6729dfbc08ef admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:36.609131 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1439/2850] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:29:36.613149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2ba8a-4094-4137-b6e3-04d50114928b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:36.623160 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2ba8a-4094-4137-b6e3-04d50114928b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:36.627699 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca50d7-284f-4121-8af0-9b26dd5dd616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:36.628357 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca50d7-284f-4121-8af0-9b26dd5dd616 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:36.628640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca50d7-284f-4121-8af0-9b26dd5dd616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:36.628916 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca50d7-284f-4121-8af0-9b26dd5dd616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:36.629634 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca50d7-284f-4121-8af0-9b26dd5dd616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:36.631250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca50d7-284f-4121-8af0-9b26dd5dd616 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:36.641299 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1412/2851] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:29:36.645624 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267777e3-95cb-420a-a315-6729dfbc08ef admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:36.657187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267777e3-95cb-420a-a315-6729dfbc08ef admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:36.662150 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc33315-1f1c-4898-988c-81edb50aaf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:36.663806 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc33315-1f1c-4898-988c-81edb50aaf3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:36.664085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc33315-1f1c-4898-988c-81edb50aaf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:36.664358 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc33315-1f1c-4898-988c-81edb50aaf3d admin admin] PATH_INFO: `/v3/projects/cfe6a9374c6d48e9933d6f099cc32bd2/users/246d8b08525243c893175a0aff2dabff/roles/7f6dd280cf8941dcb8f79ecf88dbda78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:36.681017 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc33315-1f1c-4898-988c-81edb50aaf3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe6a9374c6d48e9933d6f099cc32bd2, user_id=246d8b08525243c893175a0aff2dabff, role_id=7f6dd280cf8941dcb8f79ecf88dbda78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:36.683752 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc33315-1f1c-4898-988c-81edb50aaf3d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:36.697371 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1440/2852] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:29:36 2022] PUT /identity/v3/projects/cfe6a9374c6d48e9933d6f099cc32bd2/users/246d8b08525243c893175a0aff2dabff/roles/7f6dd280cf8941dcb8f79ecf88dbda78 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:29:36.702969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff885f83-376e-4187-8689-1d09f06f944d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:36.703324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff885f83-376e-4187-8689-1d09f06f944d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:36.703692 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff885f83-376e-4187-8689-1d09f06f944d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:36.722719 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ff885f83-376e-4187-8689-1d09f06f944d None None] MFA Rules not processed for user `cada5de94a8d4bee93afa64a6b5b9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:36.723918 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ff885f83-376e-4187-8689-1d09f06f944d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:36.729035 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:36.791685 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1413/2853] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:36.802003 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc33315-1f1c-4898-988c-81edb50aaf3d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:36.803753 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3dc33315-1f1c-4898-988c-81edb50aaf3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:36.861569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc33315-1f1c-4898-988c-81edb50aaf3d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:36.865535 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f601aa8-2936-4415-b395-927fede0bbe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:36.866215 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f601aa8-2936-4415-b395-927fede0bbe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:36.866504 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f601aa8-2936-4415-b395-927fede0bbe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:36.866786 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f601aa8-2936-4415-b395-927fede0bbe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:36.876406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f601aa8-2936-4415-b395-927fede0bbe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:36.877823 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f601aa8-2936-4415-b395-927fede0bbe8 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:36.887202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1441/2854] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:29:41.354038 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af75b4f-716e-4d04-85ec-ca497a1b620f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:41.354557 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af75b4f-716e-4d04-85ec-ca497a1b620f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:41.354671 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af75b4f-716e-4d04-85ec-ca497a1b620f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:41.385349 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8af75b4f-716e-4d04-85ec-ca497a1b620f None None] MFA Rules not processed for user `246d8b08525243c893175a0aff2dabff`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:41.386612 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8af75b4f-716e-4d04-85ec-ca497a1b620f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:41.483744 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1414/2855] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:41.488459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f601aa8-2936-4415-b395-927fede0bbe8 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:41.498247 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f601aa8-2936-4415-b395-927fede0bbe8 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:41.504025 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b52201-ad10-49d4-a198-839df5141b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:41.505157 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b52201-ad10-49d4-a198-839df5141b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:41.505505 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b52201-ad10-49d4-a198-839df5141b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:41.506009 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b52201-ad10-49d4-a198-839df5141b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:41.506694 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b52201-ad10-49d4-a198-839df5141b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:41.508835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b52201-ad10-49d4-a198-839df5141b50 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:41.550312 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1442/2856] 10.222.0.42 () {60 vars in 1152 bytes} [Wed May 25 16:29:41 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:29:41.553993 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af75b4f-716e-4d04-85ec-ca497a1b620f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:41.566095 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af75b4f-716e-4d04-85ec-ca497a1b620f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:41.648637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bb763-2fdf-4c20-a90e-bbdbf6275ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:41.649832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bb763-2fdf-4c20-a90e-bbdbf6275ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:41.650300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bb763-2fdf-4c20-a90e-bbdbf6275ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:41.650807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bb763-2fdf-4c20-a90e-bbdbf6275ac9 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:41.651923 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bb763-2fdf-4c20-a90e-bbdbf6275ac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:41.655617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bb763-2fdf-4c20-a90e-bbdbf6275ac9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:41.687852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1415/2857] 10.222.0.42 () {60 vars in 1143 bytes} [Wed May 25 16:29:41 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:29:41.690024 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b52201-ad10-49d4-a198-839df5141b50 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:41.706840 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b52201-ad10-49d4-a198-839df5141b50 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:41.715154 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d9ebd-5f11-4b20-9c3c-08a9fbee9850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:41.716121 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d9ebd-5f11-4b20-9c3c-08a9fbee9850 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:41.716569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d9ebd-5f11-4b20-9c3c-08a9fbee9850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:41.716993 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d9ebd-5f11-4b20-9c3c-08a9fbee9850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:41.718178 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d9ebd-5f11-4b20-9c3c-08a9fbee9850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:41.720926 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d9ebd-5f11-4b20-9c3c-08a9fbee9850 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:41.739825 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1443/2858] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:29:41.744043 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bb763-2fdf-4c20-a90e-bbdbf6275ac9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:41.755755 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bb763-2fdf-4c20-a90e-bbdbf6275ac9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:41.761395 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c53bc-a54e-4760-bfbb-4616b20b4811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:41.762339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c53bc-a54e-4760-bfbb-4616b20b4811 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:41.762617 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c53bc-a54e-4760-bfbb-4616b20b4811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:41.762888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c53bc-a54e-4760-bfbb-4616b20b4811 admin admin] PATH_INFO: `/v3/projects/21ae233aa269416bbd9a88eb2b41718b/users/98f9a0f853114bd48f4ff9715294445c/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:41.781463 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c53bc-a54e-4760-bfbb-4616b20b4811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ae233aa269416bbd9a88eb2b41718b, user_id=98f9a0f853114bd48f4ff9715294445c, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:41.784238 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c53bc-a54e-4760-bfbb-4616b20b4811 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:41.804119 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1416/2859] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:29:41 2022] PUT /identity/v3/projects/21ae233aa269416bbd9a88eb2b41718b/users/98f9a0f853114bd48f4ff9715294445c/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:29:41.806896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d9ebd-5f11-4b20-9c3c-08a9fbee9850 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:41.807807 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c53bc-a54e-4760-bfbb-4616b20b4811 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:41.809581 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7a8c53bc-a54e-4760-bfbb-4616b20b4811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:41.820929 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:41.831813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d9ebd-5f11-4b20-9c3c-08a9fbee9850 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:41.900206 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c53bc-a54e-4760-bfbb-4616b20b4811 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:41.902594 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:41.905622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92a4b9-7b32-462e-bd78-4ea1374e7bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:41.906622 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92a4b9-7b32-462e-bd78-4ea1374e7bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:41.907115 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92a4b9-7b32-462e-bd78-4ea1374e7bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:41.907460 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92a4b9-7b32-462e-bd78-4ea1374e7bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:41.910088 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b92a4b9-7b32-462e-bd78-4ea1374e7bd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:41.947029 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb3090-8dc5-467a-a225-19fb924d8be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:41.948827 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb3090-8dc5-467a-a225-19fb924d8be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:41.949114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb3090-8dc5-467a-a225-19fb924d8be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:41.949393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb3090-8dc5-467a-a225-19fb924d8be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:41.950243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb3090-8dc5-467a-a225-19fb924d8be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:41.952836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb3090-8dc5-467a-a225-19fb924d8be7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:41.978561 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1444/2860] 10.222.0.42 () {58 vars in 1121 bytes} [Wed May 25 16:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:29:41.983979 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb3090-8dc5-467a-a225-19fb924d8be7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:41.995477 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb3090-8dc5-467a-a225-19fb924d8be7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:42.001803 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945862d-52a3-4fdb-93f1-e660d0422a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:42.002914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945862d-52a3-4fdb-93f1-e660d0422a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:42.002914 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945862d-52a3-4fdb-93f1-e660d0422a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:42.003346 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945862d-52a3-4fdb-93f1-e660d0422a6e admin admin] PATH_INFO: `/v3/projects/21ae233aa269416bbd9a88eb2b41718b/users/98f9a0f853114bd48f4ff9715294445c/roles/d38fc08199294a1e9ed590da019eaf86` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:42.005895 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945862d-52a3-4fdb-93f1-e660d0422a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ae233aa269416bbd9a88eb2b41718b, user_id=98f9a0f853114bd48f4ff9715294445c, role_id=d38fc08199294a1e9ed590da019eaf86)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:42.009205 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945862d-52a3-4fdb-93f1-e660d0422a6e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:42.015454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92a4b9-7b32-462e-bd78-4ea1374e7bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:42.017359 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92a4b9-7b32-462e-bd78-4ea1374e7bd3 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:42.026234 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1445/2861] 10.222.0.42 () {60 vars in 1482 bytes} [Wed May 25 16:29:41 2022] PUT /identity/v3/projects/21ae233aa269416bbd9a88eb2b41718b/users/98f9a0f853114bd48f4ff9715294445c/roles/d38fc08199294a1e9ed590da019eaf86 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:29:42.030077 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1417/2862] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:29:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:29:49.502777 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdcbb6-f1f1-4992-8606-d7a3517d8da0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:49.504169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdcbb6-f1f1-4992-8606-d7a3517d8da0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:49.504706 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdcbb6-f1f1-4992-8606-d7a3517d8da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:49.541181 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4dbdcbb6-f1f1-4992-8606-d7a3517d8da0 None None] MFA Rules not processed for user `98f9a0f853114bd48f4ff9715294445c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:49.542898 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4dbdcbb6-f1f1-4992-8606-d7a3517d8da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:49.663617 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1446/2863] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:29:49.682092 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92a4b9-7b32-462e-bd78-4ea1374e7bd3 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:29:49.684090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b92a4b9-7b32-462e-bd78-4ea1374e7bd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:49.864598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92a4b9-7b32-462e-bd78-4ea1374e7bd3 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:29:49.986781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:29:50.003093 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145c485-8f66-4704-b8f5-4eaff8a41381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:29:50.003984 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145c485-8f66-4704-b8f5-4eaff8a41381 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:50.004116 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145c485-8f66-4704-b8f5-4eaff8a41381 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:50.004324 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145c485-8f66-4704-b8f5-4eaff8a41381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:50.069204 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145c485-8f66-4704-b8f5-4eaff8a41381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:29:50.073791 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145c485-8f66-4704-b8f5-4eaff8a41381 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:29:50.084841 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1418/2864] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:29:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:29:51.070545 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f4c06-a35b-445a-9e53-8b09bc1d9a07 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:29:51.071031 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f4c06-a35b-445a-9e53-8b09bc1d9a07 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:29:51.071167 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f4c06-a35b-445a-9e53-8b09bc1d9a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:29:51.106415 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-939f4c06-a35b-445a-9e53-8b09bc1d9a07 None None] MFA Rules not processed for user `246d8b08525243c893175a0aff2dabff`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:29:51.108419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-939f4c06-a35b-445a-9e53-8b09bc1d9a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:29:51.115269 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1447/2865] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:29:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:30:00.079569 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=31 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:30:01.094882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=54,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:30:34.842473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145c485-8f66-4704-b8f5-4eaff8a41381 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:30:34.944366 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145c485-8f66-4704-b8f5-4eaff8a41381 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:30:34.947342 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014238a-079a-41e0-9754-2ef66b3fb94d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:30:34.948104 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014238a-079a-41e0-9754-2ef66b3fb94d service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:34.948492 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014238a-079a-41e0-9754-2ef66b3fb94d service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:34.956747 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014238a-079a-41e0-9754-2ef66b3fb94d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:34.959837 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1014238a-079a-41e0-9754-2ef66b3fb94d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:34.979271 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014238a-079a-41e0-9754-2ef66b3fb94d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:30:34.980781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014238a-079a-41e0-9754-2ef66b3fb94d service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:30:34.996485 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1419/2866] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:30:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:30:36.194610 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f4c06-a35b-445a-9e53-8b09bc1d9a07 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:30:36.215165 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f4c06-a35b-445a-9e53-8b09bc1d9a07 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:30:36.222418 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba91c08-d6d7-4215-b8e5-db0fb8c0e2d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:30:36.223169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba91c08-d6d7-4215-b8e5-db0fb8c0e2d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:36.223999 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba91c08-d6d7-4215-b8e5-db0fb8c0e2d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:36.224342 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba91c08-d6d7-4215-b8e5-db0fb8c0e2d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:36.226419 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ba91c08-d6d7-4215-b8e5-db0fb8c0e2d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:36.281156 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014238a-079a-41e0-9754-2ef66b3fb94d service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:30:36.292945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014238a-079a-41e0-9754-2ef66b3fb94d service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:30:36.299781 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492411f8-05da-4f28-98e6-8f983bb97da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:30:36.300912 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492411f8-05da-4f28-98e6-8f983bb97da4 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:36.301197 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492411f8-05da-4f28-98e6-8f983bb97da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:36.301481 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492411f8-05da-4f28-98e6-8f983bb97da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:36.303505 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-492411f8-05da-4f28-98e6-8f983bb97da4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:36.327898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492411f8-05da-4f28-98e6-8f983bb97da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:30:36.330672 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492411f8-05da-4f28-98e6-8f983bb97da4 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:30:36.338593 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba91c08-d6d7-4215-b8e5-db0fb8c0e2d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:30:36.341202 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba91c08-d6d7-4215-b8e5-db0fb8c0e2d4 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:30:36.352681 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1420/2867] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:30:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:30:36.355642 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1448/2868] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:30:40.889680 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492411f8-05da-4f28-98e6-8f983bb97da4 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:30:40.899741 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492411f8-05da-4f28-98e6-8f983bb97da4 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:30:40.906334 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465a9fd-2e12-4bc4-9ad2-67739341fa60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:30:40.907282 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465a9fd-2e12-4bc4-9ad2-67739341fa60 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:40.907801 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465a9fd-2e12-4bc4-9ad2-67739341fa60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:40.908313 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465a9fd-2e12-4bc4-9ad2-67739341fa60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:40.924302 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465a9fd-2e12-4bc4-9ad2-67739341fa60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:30:40.926410 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465a9fd-2e12-4bc4-9ad2-67739341fa60 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:30:40.940632 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1421/2869] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:30:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:30:43.900011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba91c08-d6d7-4215-b8e5-db0fb8c0e2d4 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:30:43.901791 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ba91c08-d6d7-4215-b8e5-db0fb8c0e2d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:43.940426 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba91c08-d6d7-4215-b8e5-db0fb8c0e2d4 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:30:43.945814 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:30:43.946713 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:43.946998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:43.949855 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] PATH_INFO: `/v3/users/c935f035c757482ca912a8c4b77714a0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:43.957958 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c935f035c757482ca912a8c4b77714a0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:30:43.960278 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:30:44.042273 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfbe6b-638d-459d-b8e2-b65424ffdf4c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:44.043319 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfbe6b-638d-459d-b8e2-b65424ffdf4c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:44.044384 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfbe6b-638d-459d-b8e2-b65424ffdf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:44.052460 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c935f035c757482ca912a8c4b77714a0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.053704 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c935f035c757482ca912a8c4b77714a0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.054353 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c935f035c757482ca912a8c4b77714a0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.073260 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c935f035c757482ca912a8c4b77714a0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.080825 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9ccfbe6b-638d-459d-b8e2-b65424ffdf4c None None] MFA Rules not processed for user `28d0cf08accf4a28b85ac2342d9c3023`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:30:44.082607 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ccfbe6b-638d-459d-b8e2-b65424ffdf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:44.084384 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c935f035c757482ca912a8c4b77714a0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.128021 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1449/2870] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:30:43 2022] DELETE /identity/v3/users/c935f035c757482ca912a8c4b77714a0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:30:44.240350 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1422/2871] 10.222.0.42 () {58 vars in 957 bytes} [Wed May 25 16:30:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:30:44.309066 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:30:44.311998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:44.441468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a11a3c-a891-4eea-a99f-8c0b186173aa admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:30:44.457250 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfbe6b-638d-459d-b8e2-b65424ffdf4c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:30:44.459291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95595bd4-d565-4f53-8955-e58e70735f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:30:44.459867 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95595bd4-d565-4f53-8955-e58e70735f42 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:44.460258 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95595bd4-d565-4f53-8955-e58e70735f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:44.460641 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95595bd4-d565-4f53-8955-e58e70735f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:44.462137 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ccfbe6b-638d-459d-b8e2-b65424ffdf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:44.462964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-95595bd4-d565-4f53-8955-e58e70735f42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:44.491873 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95595bd4-d565-4f53-8955-e58e70735f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:30:44.494461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfbe6b-638d-459d-b8e2-b65424ffdf4c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:30:44.504123 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95595bd4-d565-4f53-8955-e58e70735f42 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:30:44.524468 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1450/2872] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:30:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:30:44.604229 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:30:44.605103 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:44.605390 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:44.605666 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] PATH_INFO: `/v3/projects/a40431f830294ea0a6395be072560dea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:44.607648 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a40431f830294ea0a6395be072560dea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:30:44.609584 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:30:44.647364 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:30:44.650002 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95595bd4-d565-4f53-8955-e58e70735f42 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:30:44.663191 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95595bd4-d565-4f53-8955-e58e70735f42 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:30:44.671552 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7694fb4-d141-49b0-a887-7f2c47a21bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:30:44.681970 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7694fb4-d141-49b0-a887-7f2c47a21bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:44.682569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7694fb4-d141-49b0-a887-7f2c47a21bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:44.683148 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7694fb4-d141-49b0-a887-7f2c47a21bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:44.688252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e7694fb4-d141-49b0-a887-7f2c47a21bb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:44.694721 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a40431f830294ea0a6395be072560dea', 'request_id': 'req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.705930 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:30:44.710513 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7694fb4-d141-49b0-a887-7f2c47a21bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:30:44.714174 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7694fb4-d141-49b0-a887-7f2c47a21bb3 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:30:44.714708 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a40431f830294ea0a6395be072560dea', 'request_id': 'req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.730635 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a40431f830294ea0a6395be072560dea', 'request_id': 'req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.749013 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] The token cache is being invalidate because project a40431f830294ea0a6395be072560dea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:30:44.749507 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.750459 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.755367 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1423/2873] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:30:44 2022] DELETE /identity/v3/projects/a40431f830294ea0a6395be072560dea => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:30:44.756800 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:30:44.758558 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:44.768268 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1451/2874] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:30:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:30:44.875646 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb82af3-8fd2-40f4-bda5-3cc1195e2e1b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:30:44.881969 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:30:44.882986 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:44.883582 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:44.883973 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] PATH_INFO: `/v3/users/5fbc96d1894e4a9caa79278947fb018d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:44.885295 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fbc96d1894e4a9caa79278947fb018d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:30:44.888980 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:30:44.978795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fbc96d1894e4a9caa79278947fb018d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.993240 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fbc96d1894e4a9caa79278947fb018d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:44.994052 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fbc96d1894e4a9caa79278947fb018d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:45.010089 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fbc96d1894e4a9caa79278947fb018d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:45.022171 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fbc96d1894e4a9caa79278947fb018d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:45.027237 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1424/2875] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:30:44 2022] DELETE /identity/v3/users/5fbc96d1894e4a9caa79278947fb018d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:30:45.237632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7694fb4-d141-49b0-a887-7f2c47a21bb3 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:30:45.268040 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7694fb4-d141-49b0-a887-7f2c47a21bb3 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:30:45.277852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:30:45.281705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:45.324472 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:30:45.371454 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994c8ed-3df7-4db3-98ab-363277b00e2b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:30:45.376946 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b2c2c-2e81-4f1d-b6c8-0f2e055bcdb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:30:45.377595 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b2c2c-2e81-4f1d-b6c8-0f2e055bcdb1 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:45.377835 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:30:45.377987 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b2c2c-2e81-4f1d-b6c8-0f2e055bcdb1 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:45.378175 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b2c2c-2e81-4f1d-b6c8-0f2e055bcdb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:45.378757 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:30:45.379032 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:30:45.379322 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] PATH_INFO: `/v3/projects/4a7c7b30b9bd4821a75beca185c91028` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:30:45.380061 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-191b2c2c-2e81-4f1d-b6c8-0f2e055bcdb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:30:45.380536 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a7c7b30b9bd4821a75beca185c91028)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:30:45.382483 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:30:45.393172 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b2c2c-2e81-4f1d-b6c8-0f2e055bcdb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:30:45.394990 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b2c2c-2e81-4f1d-b6c8-0f2e055bcdb1 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:30:45.406888 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1425/2876] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:30:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:30:45.447632 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a7c7b30b9bd4821a75beca185c91028', 'request_id': 'req-d639cef8-1fa5-40a4-be5b-681375491d35'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:45.455398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a7c7b30b9bd4821a75beca185c91028', 'request_id': 'req-d639cef8-1fa5-40a4-be5b-681375491d35'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:45.464000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a7c7b30b9bd4821a75beca185c91028', 'request_id': 'req-d639cef8-1fa5-40a4-be5b-681375491d35'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:45.476672 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] The token cache is being invalidate because project 4a7c7b30b9bd4821a75beca185c91028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:30:45.476965 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:45.477607 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d639cef8-1fa5-40a4-be5b-681375491d35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:30:45.490300 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1452/2877] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:30:45 2022] DELETE /identity/v3/projects/4a7c7b30b9bd4821a75beca185c91028 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:30:55.401169 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=18 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:30:55.474847 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:31:12.847187 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b2c2c-2e81-4f1d-b6c8-0f2e055bcdb1 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:31:12.848702 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-191b2c2c-2e81-4f1d-b6c8-0f2e055bcdb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:31:12.948261 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b2c2c-2e81-4f1d-b6c8-0f2e055bcdb1 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:31:13.056339 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe361c-ab96-44ea-980e-87896f45156b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:31:13.057596 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe361c-ab96-44ea-980e-87896f45156b service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:31:13.058217 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe361c-ab96-44ea-980e-87896f45156b service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:31:13.058840 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe361c-ab96-44ea-980e-87896f45156b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:31:13.062415 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2cbe361c-ab96-44ea-980e-87896f45156b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:31:13.091370 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe361c-ab96-44ea-980e-87896f45156b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:31:13.094624 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe361c-ab96-44ea-980e-87896f45156b service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:31:13.107396 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1426/2878] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:31:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:31:22.156541 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590dec1-611c-4f31-be6e-929e88fd8b11 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:31:22.157076 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590dec1-611c-4f31-be6e-929e88fd8b11 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:31:22.157486 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590dec1-611c-4f31-be6e-929e88fd8b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:31:22.182194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3590dec1-611c-4f31-be6e-929e88fd8b11 None None] MFA Rules not processed for user `730cb89f7a214fde9c5040674323f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:31:22.182194 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3590dec1-611c-4f31-be6e-929e88fd8b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:31:22.241518 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1453/2879] 10.222.0.42 () {58 vars in 967 bytes} [Wed May 25 16:31:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:31:22.291992 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe361c-ab96-44ea-980e-87896f45156b service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:31:22.294173 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2cbe361c-ab96-44ea-980e-87896f45156b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:31:22.362705 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe361c-ab96-44ea-980e-87896f45156b service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:31:22.367892 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75727f64-9233-4923-a4ae-e06547cff5f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6eb13611d0d4b6abe035c9bfbfde79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:31:22.368552 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75727f64-9233-4923-a4ae-e06547cff5f0 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:31:22.368848 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75727f64-9233-4923-a4ae-e06547cff5f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:31:22.369157 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75727f64-9233-4923-a4ae-e06547cff5f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:31:22.377609 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75727f64-9233-4923-a4ae-e06547cff5f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:31:22.379216 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75727f64-9233-4923-a4ae-e06547cff5f0 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:31:22.387668 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1427/2880] 10.222.0.42 () {60 vars in 1384 bytes} [Wed May 25 16:31:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:31:29.169945 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590dec1-611c-4f31-be6e-929e88fd8b11 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:31:29.171998 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3590dec1-611c-4f31-be6e-929e88fd8b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:31:29.242440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590dec1-611c-4f31-be6e-929e88fd8b11 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:31:29.247060 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:31:29.247933 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:31:29.248222 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:31:29.248500 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] PATH_INFO: `/v3/users/fcca87cd01ae464cb1484c3bbd52aaf6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:31:29.249746 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcca87cd01ae464cb1484c3bbd52aaf6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:31:29.251676 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:31:29.308741 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcca87cd01ae464cb1484c3bbd52aaf6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.309443 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcca87cd01ae464cb1484c3bbd52aaf6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.310069 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcca87cd01ae464cb1484c3bbd52aaf6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.322462 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcca87cd01ae464cb1484c3bbd52aaf6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.330018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcca87cd01ae464cb1484c3bbd52aaf6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.344440 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1454/2881] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:31:29 2022] DELETE /identity/v3/users/fcca87cd01ae464cb1484c3bbd52aaf6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:31:29.594819 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75727f64-9233-4923-a4ae-e06547cff5f0 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:31:29.611935 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75727f64-9233-4923-a4ae-e06547cff5f0 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:31:29.622536 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:31:29.680227 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:31:29.681105 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:31:29.681386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:31:29.681660 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] PATH_INFO: `/v3/projects/c8efc0621a1242329970cda13b3fee4b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:31:29.682878 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8efc0621a1242329970cda13b3fee4b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:31:29.685593 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:31:29.737073 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8efc0621a1242329970cda13b3fee4b', 'request_id': 'req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.742544 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8efc0621a1242329970cda13b3fee4b', 'request_id': 'req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.749501 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8efc0621a1242329970cda13b3fee4b', 'request_id': 'req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.761298 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] The token cache is being invalidate because project c8efc0621a1242329970cda13b3fee4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:31:29.762328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.763080 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.764818 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1428/2882] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:31:29 2022] DELETE /identity/v3/projects/c8efc0621a1242329970cda13b3fee4b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:31:29.769044 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:31:29.770495 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:31:29.783239 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6472de4-3efe-4965-ac16-e0baaf97608b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:31:29.804085 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:31:29.843286 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:31:29.844078 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:31:29.844360 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:31:29.844630 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] PATH_INFO: `/v3/users/a8fb914d98984987b928e79d4b75aa4d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:31:29.845877 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8fb914d98984987b928e79d4b75aa4d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:31:29.847813 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:31:29.899006 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8fb914d98984987b928e79d4b75aa4d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.899655 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8fb914d98984987b928e79d4b75aa4d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.900251 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8fb914d98984987b928e79d4b75aa4d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.916252 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8fb914d98984987b928e79d4b75aa4d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.923318 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8fb914d98984987b928e79d4b75aa4d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:29.935836 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1455/2883] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:31:29 2022] DELETE /identity/v3/users/a8fb914d98984987b928e79d4b75aa4d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:31:30.103540 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:31:30.105120 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:31:30.118125 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d091fc2-1442-44c9-ab36-1fb3d0cdc81f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:31:30.177300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:31:30.178191 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:31:30.178473 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:31:30.178745 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] PATH_INFO: `/v3/projects/ac6390639b1e42bdbdb83766fea23486` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:31:30.179902 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac6390639b1e42bdbdb83766fea23486)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:31:30.181852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:31:30.225149 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:31:30.228737 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac6390639b1e42bdbdb83766fea23486', 'request_id': 'req-29734656-e5d7-44db-a1bf-7b888cfbb3c7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:30.235328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac6390639b1e42bdbdb83766fea23486', 'request_id': 'req-29734656-e5d7-44db-a1bf-7b888cfbb3c7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:30.241439 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac6390639b1e42bdbdb83766fea23486', 'request_id': 'req-29734656-e5d7-44db-a1bf-7b888cfbb3c7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:30.253823 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] The token cache is being invalidate because project ac6390639b1e42bdbdb83766fea23486 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:31:30.254114 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:30.254761 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:30.256333 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1429/2884] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:31:30 2022] DELETE /identity/v3/projects/ac6390639b1e42bdbdb83766fea23486 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:31:39.931498 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:31:40.250504 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:31:44.139494 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:31:44.140935 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:31:44.154677 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033a91a-7c05-47cd-9816-799ec69f0ad5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:31:44.212018 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79515c-c669-4eb2-8f33-b63b8cc0c6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:31:44.212887 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79515c-c669-4eb2-8f33-b63b8cc0c6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:31:44.213169 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79515c-c669-4eb2-8f33-b63b8cc0c6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:31:44.213445 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79515c-c669-4eb2-8f33-b63b8cc0c6ac admin admin] PATH_INFO: `/v3/users/28d0cf08accf4a28b85ac2342d9c3023` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:31:44.214684 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79515c-c669-4eb2-8f33-b63b8cc0c6ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d0cf08accf4a28b85ac2342d9c3023)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:31:44.216607 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79515c-c669-4eb2-8f33-b63b8cc0c6ac admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:31:44.262397 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e79515c-c669-4eb2-8f33-b63b8cc0c6ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d0cf08accf4a28b85ac2342d9c3023'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:44.263024 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e79515c-c669-4eb2-8f33-b63b8cc0c6ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d0cf08accf4a28b85ac2342d9c3023'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:44.263639 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e79515c-c669-4eb2-8f33-b63b8cc0c6ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d0cf08accf4a28b85ac2342d9c3023'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:44.275333 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e79515c-c669-4eb2-8f33-b63b8cc0c6ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d0cf08accf4a28b85ac2342d9c3023'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:44.283511 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e79515c-c669-4eb2-8f33-b63b8cc0c6ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d0cf08accf4a28b85ac2342d9c3023'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:44.297510 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1456/2885] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:31:44 2022] DELETE /identity/v3/users/28d0cf08accf4a28b85ac2342d9c3023 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:31:44.469926 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:31:44.471411 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:31:44.484650 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29734656-e5d7-44db-a1bf-7b888cfbb3c7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:31:44.543340 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:31:44.544300 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:31:44.544598 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:31:44.544852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] PATH_INFO: `/v3/projects/9f6a5a6ae8b2488f8a5af6d2066fa88e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:31:44.546062 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f6a5a6ae8b2488f8a5af6d2066fa88e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:31:44.547945 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:31:44.593554 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f6a5a6ae8b2488f8a5af6d2066fa88e', 'request_id': 'req-c67de772-f88a-4867-b87f-75deb3075f6a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:44.599106 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f6a5a6ae8b2488f8a5af6d2066fa88e', 'request_id': 'req-c67de772-f88a-4867-b87f-75deb3075f6a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:44.605706 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f6a5a6ae8b2488f8a5af6d2066fa88e', 'request_id': 'req-c67de772-f88a-4867-b87f-75deb3075f6a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:44.618176 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] The token cache is being invalidate because project 9f6a5a6ae8b2488f8a5af6d2066fa88e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:31:44.618466 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:44.619090 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c67de772-f88a-4867-b87f-75deb3075f6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:31:44.621328 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1430/2886] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:31:44 2022] DELETE /identity/v3/projects/9f6a5a6ae8b2488f8a5af6d2066fa88e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:31:54.294000 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:31:54.615132 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:02.259398 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f4e18-30e5-4863-841b-1e79c0d2b813 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:33:02.263406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f4e18-30e5-4863-841b-1e79c0d2b813 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:33:02.263406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f4e18-30e5-4863-841b-1e79c0d2b813 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:33:02.263406 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1457/2887] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:33:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:33:02.269656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafd8c1-9a8e-4cb4-9353-40e113c8998f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:33:02.270167 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafd8c1-9a8e-4cb4-9353-40e113c8998f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:33:02.270417 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafd8c1-9a8e-4cb4-9353-40e113c8998f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:33:02.342479 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0cafd8c1-9a8e-4cb4-9353-40e113c8998f None None] MFA Rules not processed for user `e42c796dbd7441289a7cbc328abeaf81`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:33:02.343709 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0cafd8c1-9a8e-4cb4-9353-40e113c8998f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:33:02.423309 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1431/2888] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:33:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:33:02.437348 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f4e18-30e5-4863-841b-1e79c0d2b813 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:33:02.439134 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2a6f4e18-30e5-4863-841b-1e79c0d2b813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:33:02.452439 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f4e18-30e5-4863-841b-1e79c0d2b813 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:33:02.455906 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f90cc9-204c-49c1-b6b5-a5cca6c4f1d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:33:02.456537 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f90cc9-204c-49c1-b6b5-a5cca6c4f1d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:33:02.456822 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f90cc9-204c-49c1-b6b5-a5cca6c4f1d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:33:02.457099 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f90cc9-204c-49c1-b6b5-a5cca6c4f1d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:33:02.464429 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f90cc9-204c-49c1-b6b5-a5cca6c4f1d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:33:02.465896 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f90cc9-204c-49c1-b6b5-a5cca6c4f1d0 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:33:02.473560 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1458/2889] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:33:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:33:02.555268 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fb04-ba2d-44e7-9077-797b256e4c62 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:33:02.555561 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fb04-ba2d-44e7-9077-797b256e4c62 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:33:02.555836 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fb04-ba2d-44e7-9077-797b256e4c62 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:33:02.558049 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1432/2890] 10.222.0.42 () {54 vars in 921 bytes} [Wed May 25 16:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:33:02.563549 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80799f42-1954-463b-9962-d52dbe0fe889 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:33:02.563843 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80799f42-1954-463b-9962-d52dbe0fe889 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:33:02.564114 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80799f42-1954-463b-9962-d52dbe0fe889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:33:02.583294 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-80799f42-1954-463b-9962-d52dbe0fe889 None None] MFA Rules not processed for user `6e6795b5b5cf4499ad494b875756ba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:33:02.584482 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-80799f42-1954-463b-9962-d52dbe0fe889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:33:02.671102 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1459/2891] 10.222.0.42 () {58 vars in 1018 bytes} [Wed May 25 16:33:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:33:02.681637 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61fb04-ba2d-44e7-9077-797b256e4c62 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:33:02.683354 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3a61fb04-ba2d-44e7-9077-797b256e4c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:33:02.696717 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61fb04-ba2d-44e7-9077-797b256e4c62 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:33:02.700470 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0950fc-fe30-4725-a232-98a351960a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42c796dbd7441289a7cbc328abeaf81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:33:02.701104 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0950fc-fe30-4725-a232-98a351960a60 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:33:02.701386 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0950fc-fe30-4725-a232-98a351960a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:33:02.701661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0950fc-fe30-4725-a232-98a351960a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:33:02.711832 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0950fc-fe30-4725-a232-98a351960a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:33:02.714004 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0950fc-fe30-4725-a232-98a351960a60 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:33:02.724711 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1433/2892] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:33:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:33:05.020919 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80799f42-1954-463b-9962-d52dbe0fe889 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:33:05.022386 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-80799f42-1954-463b-9962-d52dbe0fe889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:33:05.039679 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80799f42-1954-463b-9962-d52dbe0fe889 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:33:05.046799 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad41a2-c4a5-4392-b3f7-9baf926b8270 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6795b5b5cf4499ad494b875756ba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:33:05.047459 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad41a2-c4a5-4392-b3f7-9baf926b8270 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:33:05.047745 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad41a2-c4a5-4392-b3f7-9baf926b8270 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:33:05.048047 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad41a2-c4a5-4392-b3f7-9baf926b8270 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:33:05.049882 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a5ad41a2-c4a5-4392-b3f7-9baf926b8270 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:33:05.062582 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad41a2-c4a5-4392-b3f7-9baf926b8270 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:33:05.064187 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad41a2-c4a5-4392-b3f7-9baf926b8270 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:33:05.075097 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1460/2893] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:33:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:33:12.720196 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=61 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:15.070430 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=52 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:33:29.050461 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0950fc-fe30-4725-a232-98a351960a60 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:33:29.051849 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-de0950fc-fe30-4725-a232-98a351960a60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:33:29.142640 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0950fc-fe30-4725-a232-98a351960a60 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:33:29.147667 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82d450-583a-4844-9044-c2533a2284ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:33:29.148312 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d450-583a-4844-9044-c2533a2284ee service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:33:29.148599 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d450-583a-4844-9044-c2533a2284ee service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:33:29.148879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d450-583a-4844-9044-c2533a2284ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:33:29.150690 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4a82d450-583a-4844-9044-c2533a2284ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:33:29.161939 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82d450-583a-4844-9044-c2533a2284ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:33:29.163515 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82d450-583a-4844-9044-c2533a2284ee service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:33:29.174898 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1434/2894] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:33:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:33:39.169695 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=41 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:30.250221 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad41a2-c4a5-4392-b3f7-9baf926b8270 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:34:30.263592 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad41a2-c4a5-4392-b3f7-9baf926b8270 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:34:30.268641 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd407a-50a9-4da6-96db-07934369c062 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:34:30.269290 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd407a-50a9-4da6-96db-07934369c062 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:34:30.269577 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd407a-50a9-4da6-96db-07934369c062 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:34:30.269888 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd407a-50a9-4da6-96db-07934369c062 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:34:30.271700 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-affd407a-50a9-4da6-96db-07934369c062 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:34:30.285020 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd407a-50a9-4da6-96db-07934369c062 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:34:30.286673 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd407a-50a9-4da6-96db-07934369c062 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:34:30.303915 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1461/2895] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:34:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:34:33.742696 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82d450-583a-4844-9044-c2533a2284ee service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:34:33.745068 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4a82d450-583a-4844-9044-c2533a2284ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:34:33.842281 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82d450-583a-4844-9044-c2533a2284ee service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:34:33.846795 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:34:33.847661 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:34:33.847944 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:34:33.848219 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] PATH_INFO: `/v3/users/246d8b08525243c893175a0aff2dabff` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:34:33.849377 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=246d8b08525243c893175a0aff2dabff)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:34:33.851392 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:34:33.896002 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '246d8b08525243c893175a0aff2dabff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:33.903414 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '246d8b08525243c893175a0aff2dabff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:33.904037 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '246d8b08525243c893175a0aff2dabff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:33.915879 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '246d8b08525243c893175a0aff2dabff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:33.922804 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '246d8b08525243c893175a0aff2dabff'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:33.928550 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1435/2896] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:34:33 2022] DELETE /identity/v3/users/246d8b08525243c893175a0aff2dabff => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:34:34.103200 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd407a-50a9-4da6-96db-07934369c062 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:34:34.118485 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd407a-50a9-4da6-96db-07934369c062 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:34:34.176110 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:34:34.176964 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:34:34.177244 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:34:34.177514 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] PATH_INFO: `/v3/projects/cfe6a9374c6d48e9933d6f099cc32bd2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:34:34.178774 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe6a9374c6d48e9933d6f099cc32bd2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:34:34.180732 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:34:34.225441 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe6a9374c6d48e9933d6f099cc32bd2', 'request_id': 'req-398aba7f-bada-484f-b626-ab6a38ebbe3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.231588 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe6a9374c6d48e9933d6f099cc32bd2', 'request_id': 'req-398aba7f-bada-484f-b626-ab6a38ebbe3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.238393 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe6a9374c6d48e9933d6f099cc32bd2', 'request_id': 'req-398aba7f-bada-484f-b626-ab6a38ebbe3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.248678 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] The token cache is being invalidate because project cfe6a9374c6d48e9933d6f099cc32bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:34:34.248960 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.249574 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.251127 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1462/2897] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:34:34 2022] DELETE /identity/v3/projects/cfe6a9374c6d48e9933d6f099cc32bd2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:34:34.255086 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:34:34.256456 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:34:34.269700 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98fe72-a679-4cb8-bf7b-725175f02649 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:34:34.317853 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:34.329358 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:34:34.330297 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:34:34.330578 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:34:34.330852 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] PATH_INFO: `/v3/users/98f9a0f853114bd48f4ff9715294445c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:34:34.332011 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f9a0f853114bd48f4ff9715294445c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:34:34.333955 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:34:34.379641 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f9a0f853114bd48f4ff9715294445c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.387139 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f9a0f853114bd48f4ff9715294445c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.387798 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f9a0f853114bd48f4ff9715294445c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.400739 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f9a0f853114bd48f4ff9715294445c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.407494 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f9a0f853114bd48f4ff9715294445c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.413423 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1436/2898] 10.222.0.42 () {58 vars in 1223 bytes} [Wed May 25 16:34:34 2022] DELETE /identity/v3/users/98f9a0f853114bd48f4ff9715294445c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:34:34.591316 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:34:34.592790 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:34:34.606357 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398aba7f-bada-484f-b626-ab6a38ebbe3e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:34:34.669108 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cada5de94a8d4bee93afa64a6b5b9d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55f82ef7ffb1469ea7582069b5da4934', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:34:34.670005 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:34:34.670291 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:34:34.670569 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] PATH_INFO: `/v3/projects/21ae233aa269416bbd9a88eb2b41718b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:34:34.671758 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ae233aa269416bbd9a88eb2b41718b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:34:34.673656 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:34:34.694011 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:34.720878 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ae233aa269416bbd9a88eb2b41718b', 'request_id': 'req-f35d42a9-0be0-413b-bf26-1ccb2da8872e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.727523 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ae233aa269416bbd9a88eb2b41718b', 'request_id': 'req-f35d42a9-0be0-413b-bf26-1ccb2da8872e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.735414 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ae233aa269416bbd9a88eb2b41718b', 'request_id': 'req-f35d42a9-0be0-413b-bf26-1ccb2da8872e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.745529 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] The token cache is being invalidate because project 21ae233aa269416bbd9a88eb2b41718b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:34:34.745789 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.746407 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f35d42a9-0be0-413b-bf26-1ccb2da8872e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:34:34.747937 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1463/2899] 10.222.0.42 () {58 vars in 1232 bytes} [Wed May 25 16:34:34 2022] DELETE /identity/v3/projects/21ae233aa269416bbd9a88eb2b41718b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:34:44.410024 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:34:44.743243 dv-cinder-843278-2-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:38:32.952072 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:38:32.953488 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:38:33.023658 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6b4ef-6d8f-4065-869e-0a68438e807f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:38:33.083434 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8adf0f-3c71-49ca-8051-f993dd6549c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6719421144734ef49b3809df30a45961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27d1825035a24259a01beb6b6610ab3f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:38:33.084086 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8adf0f-3c71-49ca-8051-f993dd6549c0 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:38:33.084379 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8adf0f-3c71-49ca-8051-f993dd6549c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:38:33.084656 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8adf0f-3c71-49ca-8051-f993dd6549c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:38:33.086444 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6c8adf0f-3c71-49ca-8051-f993dd6549c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:38:33.098070 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8adf0f-3c71-49ca-8051-f993dd6549c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:38:33.099643 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8adf0f-3c71-49ca-8051-f993dd6549c0 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:38:33.110470 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1437/2900] 10.222.0.42 () {60 vars in 1365 bytes} [Wed May 25 16:38:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:38:43.106194 dv-cinder-843278-2-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=58 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}